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Abstract 

Achieving  control  of  dynamic  and  complex  situations  is  always  challenging  involving 
as  it  does  the  management  of  cognitive  resources.  It  has  been  proposed  that  one  of  the 
leading  causes  of  error  in  such  dynamic  environments  is  a  generalised  tendency  to 
attempt  to  use  more  task  resources  than  one’s  cognitive  capacity  can  sustain,  termed 
the  overutilisation  of  resources  bias  (Omodei,  Wearing,  McLennan  &  Hansen,  2001). 
The  aim  of  the  present  study  was  to  explicitly  take  into  account  individual  differences 
in  cognitive  capacity  in  an  investigation  of  this  human  tendency  to  overuse  resources, 
and  its  proposed  effect  on  decision  making  efficiency.  Sixteen  participants  (i.e.,  10 
female  and  6  male)  aged  between  18  and  32  years  completed  ten  trials  of  the 
computer  simulated  forest  fire-fighting  task  Networked  Fire  Chief  (Omodei,  Taranto 
&  Wearing,  1999).  Adopting  a  repeated  measures  design,  all  participants  were 
administered  two  conditions:  one  in  which  they  were  given  the  maximum  number  of 
fire  fighting  resources  they  were  observed  to  have  been  able  to  cognitively  manage 
during  earlier  training  trials  (i.e.,  MANAGEABLE  condition);  and  one  in  which  they 
were  given  double  the  number  of  resources  supplied  in  the  MANAGEABLE 
condition  (i.e.,  EXCESS  condition).  It  was  predicted  that  a  tendency  to  overuse 
resources  in  the  EXCESS  condition  would  lead  to  cognitive  overload  with  a  higher 
experienced  mental  workload,  poorer  overall  awareness  of  the  situation,  and 
subsequently  poorer  decision  making  perfonnance  compared  to  the  MANAGEABLE 
condition.  However,  there  was  no  significant  difference  found  between  conditions  for 
any  of  these  three  variables.  Further  analyses  revealed  individual  differences  in  the 
ability  to  appropriately  adapt  to  the  overabundance  of  resources  in  the  EXCESS 
condition.  It  was  concluded  that  individual  flexibility  in  the  quality  of  strategic 
thought  allocated  to  resource  usage,  or  in  other  words,  the  degree  metacognitive 
control,  may  well  be  a  major  predictor  of  decision  making  efficiency  in  dynamic 
environments. 
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Introduction 

Dynamic  Decision  Making  in  Emergency  Situations 

Humans  generally  spend  a  great  proportion  of  their  daily  lives  engaged  in 
decision  making  activity.  Often,  the  decision  process  is  relatively  simple,  requiring  a 
single  choice  from  a  number  of  alternatives  in  order  to  achieve  a  desired  outcome 
(Brehmer,  1995).  However,  in  real-world  emergency  management  domains  such  as 
fire  fighting  and  piloting  aircraft,  the  decision  process  is  quite  complicated  (Omodei, 
McLennan,  Wearing  &  Speit,  2003).  Furthermore,  the  consequences  of  decisions 
which  are  made  are  more  serious.  Specifically,  the  decision  makers  in  such 
emergency  management  domains  must  attempt  to  control  environments  that  are 
typically  complex,  dynamic  and  opaque,  in  order  to  protect  life  and  property  (Omodei 
et  al.,  2003).  The  environments  are  complex,  as  they  demand  a  sequence  of  decisions 
and  actions  to  be  implemented  in  the  face  of  a  large  array  of  information,  ill-defined 
problems,  competing  goals,  uncertainty,  risk,  and  time  pressure  (Brehmer,  1995; 
Omodei,  Wearing,  McLennan  &  Hansen,  2001).  The  environments  are  dynamic  in 
that  they  are  subject  to  constant  change:  both  independently  of  any  decision  input,  and 
as  a  result  of  the  decision  makers’  actions  (Omodei,  Wearing,  McLennan  &  Hansen, 
2001).  Finally,  the  environments  are  opaque  in  that  the  state  and  characteristics  of  the 
situation  are  not  immediately  obvious  to  the  decision  maker,  requiring  hypothesis 
formation  and  testing  to  reveal  their  nature  (Brehmer,  1995). 

Achieving  control  of  such  dynamic  decision-making  (DDM)  environments  is  a 
formidable  task,  often  requiring  deep  cognitive  processing.  In  fact,  errors  in  decision 
making  are  inevitable  when  the  demands  of  a  DDM  task  are  simply  beyond  the 
decision  makers’  cognitive  capability  (Omodei  et  al.,  2003).  Therefore,  it  is  important 
to  investigate  the  nature  of  decision  errors  under  such  limitations,  and  the  extent  to 
which  such  errors  can  be  kept  to  a  minimum.  Such  a  research  program  can  be 
expected  to  produce  findings  that  will  not  only  contribute  to  a  better  understanding  of 
human  cognitive  functioning,  but  also  allow  for  the  identification  of  strategies  for 
assisting  people  to  optimise  decision  making  performance  in  complex  dynamic 
situations. 
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Studying  Dynamic  Decision  Making 

Until  recently,  the  study  of  decision  making  in  dynamic  environments  has 
been  problematic.  Field  investigations  have  been  criticised  because  they  do  not  allow 
for  the  controlled  and  precise  manipulation  of  relevant  variables,  and  thus  suffer  poor 
internal  validity  (Omodei  &  Wearing,  2005).  In  contrast,  laboratory  experiments  have 
been  criticised  for  their  poor  external  validity,  whereby  it  has  been  argued  they  have 
not  captured  the  true  nature  and  complexity  of  real  DDM  environments  (Brehmer, 
1993). 

Fortunately,  recent  advances  in  computer  technology  have  allowed  for  the 
creation  of  computer  simulated  microworlds,  in  which  scenarios  can  be  developed 
that  possess  the  key  characteristics  of  a  typical  DDM  environment,  including 
dynamism,  complexity,  opaqueness  and  uncertainty,  and  thereby  evoke  similar 
psychological  responses  to  those  elicited  in  naturally  occurring  environments 
(Omodei,  Wearing,  McLennan  &  Hansen,  2001).  In  bridging  the  gap  between  the 
inherent  complexity  of  field  investigations  and  the  rigor  of  laboratory  studies,  the  use 
of  computer  simulated  microworlds  has  become  the  dominant  and  certainly  most 
effective  method  for  studying  decision  making  in  dynamic  environments  (Omodei  & 
Wearing,  2005). 

One  such  computer-generated  microworld  program  is  known  as  Networked 
Fire  Chief  (NFC),  which  was  developed  by  Omodei  et  al.  (1999).  Specifically,  NFC 
allows  the  creation  of  simulated  fire-fighting  situations  resembling  those  situations 
typically  encountered  by  fire-fighters  in  reality.  Results  obtained  from  studies  using 
computer  simulated  microworlds  such  as  NFC  have  yielded  valuable  insight  into  the 
nature  of  human  information  processing 

Cognitive  Processing  Limitations 

When  attempting  to  control  DDM  environments,  decision-makers  must  attend 
to  multiple  sources  of  information,  detennine  the  meaning  and  value  of  this 
infonnation,  and  select  an  appropriate  response  (Wickens,  2001).  Unfortunately, 
humans  have  limited  attention  capacity,  which  means  they  are  limited  in  the  amount 
of  information  they  can  focus  on  and  on  the  rate  at  which  they  can  process  such 
infonnation.  In  fact,  decision  makers  in  dynamic  environments  tend  to  attend  to  items 
of  information  that  are  perceived  as  less  important,  sometimes  at  the  cost  of  actually 
missing  valuable  infonnation  (Endsley,  2000b;  Endsley  &  Rogers,  1998). 
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Subsequently,  an  important  source  of  error  in  decision  making  is  the  failure  to 
perceive  valuable  infonnation.  By  way  of  example,  Kuipers,  Kappers,  Van  Holten, 
Van  Bergen  and  Oosterveld  (1990)  investigated  fatal  errors  in  the  controlled  descent 
of  fighter  aircrafts  in  enemy  terrain,  and  found  that  channelised  attention  and 
preoccupation  with  a  potential  malfunction  were  the  most  common  causes  of  error. 

Regardless  of  which  infonnation  is  selectively  attended  to,  the  active 
processing  of  such  information  occurs  in  working  memory,  with  the  outcome  of  such 
processing  leading  to  information  being  either  consolidated  into  long  tenn  memory  or 
lost  (Endsley,  2000).  Generally,  it  has  been  argued  that  information  remains  resident 
in  working  memory  for  approximately  20  seconds  before  decay  begins  to  occur  unless 
it  is  actively  rehearsed  or  used  (Ploner,  Gaymard,  Rivaud,  Agid  &  Pierrot-Deseilligny, 
1998).  Unfortunately,  humans  are  limited  in  the  number  of  resources  they  have 
available  for  meeting  the  computational  and  storage  demands  of  information 
processing  (Just  &  Carpenter,  1992).  In  fact,  individuals  differ  in  regard  to  their 
processing  efficiency  and  speed.  Hence,  errors  can  also  occur  when  relevant 
infonnation  is  misinterpreted,  forgotten  or  lost  due  to  inadequate  processing. 

On  a  more  specific  level,  decision  errors  are  said  to  be  more  common  when 
one  has  inadequate  or  incomplete  knowledge  regarding  the  current  state  of  a  dynamic 
environment,  termed  their  situation  awareness  (SA)  (Endsley,  1996).  Hence, 
decrements  in  SA  occur  as  a  result  of  one’s  working  memory  and  therefore  attention 
capacity  limitations. 

The  Importance  of  Situation  Awareness  in  Dynamic  Decision  Making 

According  to  Omodei,  Wearing,  McLennan  and  Hansen’s  (2001)  Adaptive 
Control  Model  of  Decision  Making  in  DDM  environments,  humans  generally  allocate 
the  greater  part  of  their  cognitive  activity  to  situation  assessment  as  distinct  from 
action  planning  (an  aspect  of  metacognition).  Situation  assessment  involves 
perceiving  and  understanding  the  situation  and  therefore  leads  to  the  development  and 
maintenance  of  situation  awareness  (SA).  Endsley  (1995a)  proposed  that  SA  has  three 
distinct  levels:  the  perception  of  elements  in  the  environment  (i.e.,  level  1  SA);  the 
comprehension  of  their  meaning  (i.e.,  level  2  SA);  and  the  projection  of  their  future 
status  (i.e.,  level  3  SA).  The  three  levels  of  SA  are  hierarchical  in  the  sense  that  one’s 
SA  will  be  better  the  further  one  progresses  through  the  three  levels,  starting  at  level  1 
SA  (Endsley,  1995a). 
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Because  of  its  obvious  importance  in  aviation,  much  of  the  early  work  on  SA 
was  conducted  in  this  domain  (Endsley,  1999a).  Of  particular  relevance,  Jones  and 
Endsley  (1995)  conducted  an  extensive  study  of  SA  errors  based  on  voluntary  reports 
in  NASA’s  Aviation  Safety  Reporting  System  (ASRS)  database.  Specific  errors  in  SA 
included  difficulty  in  perceiving  a  vital  piece  of  information,  failure  to  even 
acknowledge  a  vital  piece  of  information,  attention  narrowing,  external  distractions, 
misperception  and  memory  loss  (Endsley,  1995a). 

Partly  because  individuals  differ  in  terms  of  their  attention  and  strategies  for 
dealing  with  inherent  limitations  in  working  memory  capacity,  individuals  differ  in 
their  ability  to  acquire  and  maintain  adequate  SA.  In  support  of  this,  Endsley  and 
Bolstad  (1994)  investigated  pilot  SA  in  an  air-to-air  fighter  sweep  mission  and  found 
evidence  of  differences  between  individuals  in  their  ability  to  achieve  an  adequate 
level  of  SA  given  the  same  situation.  Specifically,  the  importance  of  spatial  and 
perceptual  skills,  and  to  a  lesser  extent  attention  sharing  and  pattern  matching  were 
emphasised  as  predicting  ability  to  acquire  SA  (Endsley  &  Bolstad,  1994). 

It  is  important  to  note  that  even  a  person  with  perfect  SA  may  still  make  the 
wrong  decision,  due  to  such  factors  as  lack  of  knowledge  or  the  adoption  of  poor 
decision  strategies  (Endsley,  1995a).  Good  SA  can  therefore  be  considered  a  factor 
that  will  increase  the  probability  of  good  perfonnance,  but  cannot  necessarily 
guarantee  it  (Endsley,  1995  a).  There  is  no  set  threshold  of  SA  that  can  guarantee  a 
given  level  of  performance  (Endsley,  2000).  Nonetheless,  poor  SA  and  resultant 
decision  errors  have  been  found  to  be  more  likely  when  mental  workload  is  high. 

Mental  Workload  and  Decision  Making  Efficiency 

Almost  always  in  DDM  environments,  the  decision  maker  will  experience  a 
high  level  of  mental  workload,  given  the  quantity  and  complexity  of  demands  placed 
on  them.  When  there  is  more  information  available  than  one  can  process  at  a  given 
time,  this  is  referred  to  as  cognitive  overload  (Joslyn  &  Hunt,  1998).  Interestingly, 
Gibson,  Orasanu,  Villeda  and  Nygren  (1997)  reviewed  the  ASRS  that  contains 
information  of  aviation  major  incidents,  and  found  that  cognitive  overload  was 
amongst  the  major  predictors  of  loss  of  SA  and  subsequent  decision  error. 

In  support  of  this,  Endsley  and  Rodgers  (1998)  investigated  errors  in  air  traffic 
control  and  found  that  cognitive  overload  was  primarily  the  cause  of  decision  errors 
including  misperceptions,  and  failures  in  monitoring  and  anticipation  of  the  future 
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states.  Other  deleterious  effects  of  such  cognitive  overload  might  include  decrements 
in  maintenance  of  adequate  SA  and  impairment  of  high-level  strategic  thought 
(Omodei,  Wearing,  McLennan  &  Hansen,  2001).  Therefore,  mental  workload  has  also 
been  implicated  as  a  major  predictor  of  decision  error  in  DDM  environments.  It  is 
important  to  note  that  task  load,  defined  as  the  demand  imposed  by  a  task,  is  generally 
distinguished  from  mental  workload,  defined  as  the  controller’s  subjective  experience 
of  that  demand  (Hilburn  &  Jorna,  2001).  A  high  task  load  will  only  impose  a  heavy 
mental  workload  on  the  decision  maker  when  the  demands  of  the  task  exceed  the 
capacity  of  the  decision  maker’s  attention  and  working  memory  resources  (Endsley, 
1995a;  Just  &  Carpenter,  1992).  Furthermore,  it  is  important  to  note  that  there  are 
individual  differences  in  the  extent  to  which  a  high  task  load  will  impose  a  heavy 
mental  workload  (Brozovic,  2004;  Wickens,  2002).  This  is  because  there  are 
individual  differences  in  overall  cognitive  capacity  and  subsequent  processing 
efficiency  (Just  &  Carpenter,  1992). 

In  investigating  the  phenomena  of  cognitive  overload,  Omodei  et  al.  (2003) 
provided  participants  with  twelve  versus  twenty-eight  fire  fighting  appliances  to 
command  in  the  NFC  fire  fighting  microworld,  having  assumed  from  pilot  testing  that 
twelve  was  the  number  of  appliances  that  could  be  cognitively  managed.  It  was  found 
that  the  participants  became  significantly  cognitively  overloaded  with  twenty-eight 
appliances,  to  the  extent  that  their  decision  making  performance  was  significantly 
worse  compared  with  when  they  had  twelve  appliances  (Omodei  et  al.,  2003).  In  a 
replication  by  Omodei  and  Wearing  (2004),  similar  results  were  found,  and  it  was 
concluded  that  any  advantages  offered  by  the  availability  of  extra  resources  were 
offset  by  the  tendency  to  use  these  resources  regardless  of  their  utility.  Interestingly,  it 
has  been  argued  that  whenever  and  whatever  resource  is  made  available,  people  feel 
compelled  to  use  it,  even  when  their  cognitive  system  becomes  so  overloaded  as  to 
result  in  a  degradation  of  decision  making  performance  (Omodei,  Wearing, 

McLennan  &  Hansen,  2001).  This  phenomenon  has  been  termed  the  overutilisation 
bias. 

An  Overutilisation  of  Resources  Bias  in  Dynamic  Decision  Making 

It  has  been  argued  that  a  global  belief  shared  by  all  humans  is  that  “more  is 
better”  in  regard  to  information  and  other  decision  resources  which  explains  the 
tendency  to  overuse  such  resources  (Omodei,  Wearing,  McLennan,  Elliot  &  Clancy, 
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2001)  relative  to  their  allocation  of  other  activities.  In  regard  to  the  studies  by  Omodei 
et  al.  (2003)  and  Omodei  and  Wearing  (2004),  it  was  found  that  participants  were  not 
aware  they  were  overusing  fire  fighting  appliances  in  the  NFC  task,  and  participants 
could  not  detect  that  they  were  actually  performing  better  in  situations  with  fewer 
resources.  Furthermore,  the  overutilisation  bias  was  present  in  participants  even  after 
they  were  informed  of  the  nature  of  this  bias  and  told  that  using  fewer  resources 
would  be  more  beneficial  (Omodei  &  Wearing,  2004).  Therefore,  it  has  been  argued 
that  the  overutilisation  bias  may  well  operate  at  a  relatively  unconscious  level  and  that 
it  is  perhaps  a  deeply  embedded  cognitive  structure  (Omodei  et  al.,  2003).  In 
summary,  these  results  point  to  the  importance  of  a  failure  of  metacognition. 

A  similar  study  by  Brozovic  (2004)  involved  giving  participants  eight  (i.e., 
FEW  condition)  versus  sixteen  fire  fighting  appliances  (MANY  condition)  to 
command  in  NFC,  whereby  eight  was  assumed  to  be  the  number  of  appliances  that 
each  participant  could  cognitively  manage  in  the  particular  scenario  created  for  this 
study.  It  was  found  that  overuse  of  appliances  did  occur  in  the  MANY  condition,  and 
the  quality  of  strategic  thought  and  level  2  and  3  SA  were  lower  in  the  MANY 
condition,  with  high  mental  workload  degrading  decision  making  performance 
(Brozovic,  2004).  It  was  concluded  that  in  the  presence  of  extra  resources,  there  may 
be  a  trade-off  between  the  quantity  of  resources  used  and  the  quality  of  cognitive 
processing  associated  with  such  resource  use  (i.e.,  quantity-quality  trade-off) 

(Omodei,  Wearing,  McLennan  &  Hansen,  2001). 

As  briefly  mentioned,  the  overutilisation  bias  appears  to  be  non-specific, 
applying  to  all  types  of  resources,  whether  this  be  for  infonnation  gathering, 
opportunities  for  action  or  communication  input  (Omodei,  Wearing,  McLennan  & 
Hansen,  2001).  Using  more  resources  obviously  means  more  cognitive  effort  must  be 
expended,  which  more  often  than  not  leads  to  cognitive  overload  (Wickens,  2001). 
Therefore,  cognitive  overload  is  said  to  be,  to  an  extent,  self-imposed  (Brozovic, 

2004;  Omodei  et  al.,  2003). 

The  most  common  argument  regarding  the  cause  of  overutilisation  is  that  it 
represents  a  general  problem  in  metacognitive  control,  that  is,  one’s  ability  to  self¬ 
monitor  and  self-regulate  the  state  of  their  working  memory  (Omodei,  Wearing, 
McLennan,  Elliot  &  Clancy,  2001;  Omodei  et  al.,  2003).  Such  self-evaluative  control 
processes  are  usually  aimed  at  maximising  the  continuing  efficacy  of  a  person’s 
decisions  by  preventing  limited  working  memory  capacity  being  exceeded  and 
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ensuring  a  balance  between  situation  assessment,  intention  generation,  and  action 
selection  (Omodei,  Wearing,  McLennan,  Elliot  &  Clancy,  2001).  However,  often  the 
attempt  to  maintain  such  a  balance  leads  to  more  activities  being  undertaken  than  can 
be  processed  and  a  suboptimal  allocation  of  mental  resources  among  various  aspects 
of  the  tasks  (Brozovic,  2004;  Omodei  et  ah,  2003).  More  specifically,  it  has  been 
argued  that  decision  makers  exhibit  poor  metacognitive  control  because  they 
overestimate  their  own  cognitive  capacity,  they  feel  a  greater  sense  of  control  and 
ability  when  engaged  in  constant  activity,  and  they  prefer  errors  of  commission  over 
errors  of  omission  (Omodei  et  ah,  2003). 

In  summary,  there  are  well-documented  individual  differences  in  skill, 
experience,  goals,  expectations,  attention  capacity,  overall  cognitive  capacity  and 
processing  efficiency  (Brozovic,  2004;  Hilburn  &  Jorna,  2001;  Just  &  Carpenter, 

1992;  Parasuraman  &  Hancock,  2001).  Hence,  there  are  individual  differences  in  the 
ability  to  achieve  control  in  DDM  environments,  and  in  the  potential  for  error. 

Limitations  of  Previous  Studies  Investigating  the  Possible  Impact  of  an 
Overutilisation  of  Resources  Bias  on  Dynamic  Decision  Making  Efficiency 

In  all  previous  studies  utilizing  NFC  to  study  the  overutilisation  bias  with 
respect  to  the  number  of  appliances  available,  the  decision  making  performance, 
mental  workload  and  SA  of  participants  has  been  investigated  by  comparing  results  in 
two  conditions:  one  where  participants  were  given  a  fixed  number  of  appliances 
assumed  to  be  cognitively  manageable;  and  one  where  they  were  given  a  fixed 
number  of  assumed  over-abundance  of  appliances  (Brozovic,  2004;  Omodei  & 
Wearing,  2004;  Omodei,  Wearing,  McLennan  &  Hansen,  2001).  That  is,  all 
participants  have  been  given  the  same  number  of  appliances  in  both  appliance 
availability  conditions.  Assuming  that  all  participants  can  cognitively  manage  the 
same  number  of  appliances  is  problematic,  considering  the  well-established  individual 
differences  in  cognitive  capacity  as  discussed  above.  In  support  of  this,  Brozovic 
(2004)  concluded  that  there  appears  to  be  individual  differences  in  the  maximum 
number  of  appliances  cognitively  manageable.  Therefore,  for  adequate  investigation 
of  the  presumed  overutilisation  bias,  researchers  must  control  for  such  individual 
differences  in  the  maximum  number  of  appliances  manageable. 

Another  limitation  of  past  research  using  NFC  to  investigate  the  possible 
effects  of  the  overutilisation  bias  on  SA  has  been  in  the  operationalisation  of  SA. 
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Generally,  a  version  of  Endsley’s  (1995b)  Situation  Awareness  Global  Assessment 
Technique  (SAGAT)  has  been  used  as  an  objective  measure  of  global  SA.  This 
involves  pausing  a  simulation  at  a  time  unknwn  to  participants  and  then  asking  them 
to  complete  a  series  of  questions  regarding  their  awareness  of  the  location  of  fires,  the 
approximate  distributions  of  appliances,  fire  warnings  and  wind  conditions  (Endsley, 
1995b).  The  core  problem  with  the  SAGAT  questionnaire  is  the  time  required  for  its 
administration. 

Working  memory  is  where  dynamic  information  from  the  environment  that 
infonns  present  decision  making  would  be  resident,  so  an  SA  measure  should  tap  into 
the  contents  of  working  memory  (Endsley,  1999b).  A  significant  issue  for  measures 
that  attempt  to  tap  into  memory  of  an  essentially  non-verbal,  spatio-temporal 
awareness  is  to  what  degree  people  can  provide  a  verbal  report  on  mental  processes  to 
make  this  information  available  for  analysis  (Endsley,  1999b).  SAGAT  questionnaires 
have  generally  comprised  as  many  as  eight  questions,  and  sometimes  have  taken 
between  3-4  minutes  to  complete.  Considering  memory  decay  begins  to  occur  after 
approximately  20  seconds,  such  a  global  SA  measure  may  not  be  reliable. 
Interestingly,  all  previous  studies  using  the  SAGAT  to  measure  SA  in  NFC  have 
failed  to  find  support  for  the  hypothesised  positive  link  between  SA  and  decision 
making  efficiency,  nor  for  the  hypothesised  overlap  between  the  various  components 
of  SA.  This  may  be  because  the  SAGAT  measure  typically  used  has  been  unreliable. 

Endsley  (1995a)  has  suggested  that  pilots’  ability  to  report  their  SA  via 
SAGAT  was  unaffected  by  the  number  of  questions,  and  that  memory  decay  would 
not  be  an  issue  provided  the  administration  of  the  SAGAT  commenced  within  20 
seconds  after  a  simulation  pause.  However,  as  it  has  been  briefly  argued  above, 
infonnation  can  be  expected  to  decay  from  working  memory  after  20  seconds  (Ploner 
et  ah,  1998).  Accordingly,  the  entire  SAGAT  would  need  to  be  completed  within  20 
seconds  to  obtain  a  reliable  and  valid  measure  of  one’s  global  SA. 

The  Present  Study 

The  main  aim  of  the  present  study  was  to  re-investigate  the  possibility  of  an 
overutilisation  of  decision  resources  bias  in  NFC  generated  scenarios  by  (i) 
controlling  for  individual  differences  in  the  maximum  number  of  appliances 
manageable,  and  (ii)  using  a  version  of  the  SAGAT  that  was  able  to  be  completed 
within  20  seconds  of  the  simulation  pause.  Following  this,  the  possible  existence  of  an 
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overutilisation  of  resources  bias  and  its  true  effect  on  decision  making  performance, 
mental  workload  and  situation  awareness  could  be  investigated. 

Hypotheses 

In  the  present  study,  three  hypotheses  were  proposed: 

1)  Decision  making  performance  will  be  better  in  the  MANAGEABLE 
(relatively  few  appliances)  condition  compared  to  the  EXCESS  (relatively 
many  appliances)  condition. 

2)  Mental  Workload  will  be  lower  in  the  MANAGEABLE  condition 
compared  to  the  EXCESS  condition. 

3)  SA  will  be  better  in  the  MANAGEABLE  condition  compared  to  the 
EXCESS  condition. 

Participants 

Sixteen  volunteer  participants  (6  male  and  10  female)  were  recruited  for  this 
study,  the  majority  of  whom  were  undergraduate  students  at  La  Trobe  University.  The 
age  range  of  participants  was  18-32  years  (M  =  22.00,  SD  =  4.46).  Participants  were 
required  for  no  more  than  five  hours  each  and  were  compensated  $13  per  hour  for 
their  time.  Participants  were  naive  to  the  purpose  of  the  experiment. 
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Materials 

Dynamic  Decision  Making  Task 

The  NFC  program  was  used  in  this  study  to  create  various  simulated  dynamic 
fire-fighting  situations.  Figure  1  illustrates  one  particular  scenario  used  in  this  study  to 
create  a  DDM  environment. 
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Figure  1.  An  experimental  NFC  scenario  used  in  the  present  study. 


The  NFC  program  involves  presenting  participants  with  a  landscape  map  on 
the  computer  screen  and  instructing  them  to  control  the  spread  of  simulated  fires  using 
available  fire  fighting  appliances.  Participants  move  appliances  by  dragging  them  with 
the  mouse  and  command  the  appliances  to  take  action  by  clicking  on  them.  The 
landscape  map  includes  different  landscape  features  assigned  different  asset 
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protection  values,  and  participants  are  instructed  to  save  as  much  valuable  landscape 
as  possible.  Hence,  participants  must  consider  the  value  of  landscape  features  in 
adopting  a  strategy  for  fighting  the  fires,  whilst  also  considering  current  and  forecast 
wind  conditions,  fire  warnings  and  different  fire  spread  rates.  In  addition,  they  must 
monitor  the  water  level  of  each  appliance  and  command  the  appliances  to  refill 
whenever  necessary. 

At  the  end  of  a  NFC  trial,  the  program  provides  an  overall  perfonnance  score 
statistic  that  indicates  the  overall  value  of  landscape  assets  (as  a  percentage) 
remaining  unburnt  (Omodei,  Wearing,  McLennan  &  Hansen,  2001).  In  addition,  NFC 
provides  a  mean  idle  time  statistic  at  the  end  of  a  trial,  which  indicates  the  average 
amount  of  time  the  group  of  fire  fighting  appliances  were  left  out  of  action. 
Furthermore,  NFC  provides  a  trial  replay  of  each  task.  The  ability  of  current  computer 
simulation  programs  such  as  NFC  to  create  complex  and  realistic  DDM  environments 
is  well  documented  and  hence,  the  use  of  NFC  to  study  DDM  in  this  study  was 
warranted  (Omodei  &  Wearing,  2004). 

Dynamic  Decision  Making  Scenarios 

In  this  study,  a  total  of  eleven  NFC  trials  were  developed,  comprising  a 
demonstration  trial,  six  practice  trials  of  increasing  difficulty  and  four  experimental 
trials.  All  trials  lasted  10  minutes  each.  The  demonstration  and  practice  trials  provided 
a  chance  for  the  participants  to  gain  the  appropriate  knowledge  and  skill  needed  for 
the  NFC  program.  Furthermore,  these  trials  were  used  by  the  experimenter  to 
progressively  assess  the  maximum  number  of  fire  fighting  appliances  that  each 
participant  could  cognitively  manage.  Finally,  the  experimental  trials  were  designed 
to  assess  the  participant’s  decision-making  skill,  mental  workload  and  SA  under  the 
two  appliance  availability  conditions  (details  provided  below). 

The  scenarios  were  designed  so  that  saving  a  house  was  worth  100  points, 
while  saving  bushland  was  only  worth  1  point.  Participants  were  told  of  these  values 
and  therefore  instructed  to  strictly  prioritize  saving  houses  when  completing  a  trial. 
Participants  were  also  informed  of  the  fact  that  fires  spread  faster  through  bushland 
than  houses,  and  that  the  fires  spread  in  accordance  with  the  current  wind  direction 
and  strength. 

In  all  NFC  scenarios  used  in  previous  studies,  there  were  houses  on  the 
landscape  that  were  not  at  risk  of  being  burned.  Hence,  these  houses  remained  intact 
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even  if  a  participant  did  not  initiate  any  commands  in  the  entire  ten  minutes  of  the 
trial.  This  would  reduce  the  sensitivity  and  therefore  reliability  of  the  final  overall 
perfonnance  score  as  a  measure  of  decision  making  skill.  In  the  present  study 
therefore,  the  NFC  trials  were  designed  so  that  all  the  houses  were  at  risk  of  being 
burnt.  Hence,  the  trials  were  more  sensitive  to  differences  in  decision  making  skill. 
Specifically,  participants  who  prioritised  well  and  made  appropriate  decisions  would 
save  more  houses  than  a  less  skilled  participant,  and  would  subsequently  achieve  a 
significantly  higher  perfonnance  score. 

The  experimental  trials  were  designed  so  that  they  would  be  cognitively 
challenging  for  the  more  skilled  participants,  yet  not  overwhelmingly  difficult  for  the 
less  skilled  participants.  Participants  were  required  to  expend  substantial  mental  effort 
attending  to  multiple  critical  fires,  random  fire  warnings  and  frequent  wind  changes, 
whilst  regularly  monitoring  and  commanding  the  various  fire  fighting  appliances. 

There  were  two  experimental  conditions,  termed  the  MANAGEABLE  and 
EXCESS  conditions.  In  the  MANAGEABLE  condition,  participants  were  given  the 
maximum  number  of  fire  fighting  appliances  that  they  were  assessed  as  being  able  to 
cognitively  manage.  This  number  was  chosen  based  upon  observation  of  the 
participants  mean  idle  times  in  the  practice  trials  (see  procedure).  In  the  EXCESS 
condition,  participants  were  given  double  the  number  of  fire  fighting  appliances  they 
would  have  access  to  in  the  MANAGEABLE  condition.  This  number  of  appliances 
was  judged  to  be  substantially  more  than  the  number  that  participants  should  be  able 
to  cognitively  manage. 

The  fire-fighting  appliances  were  programmed  to  arrive  at  a  requested 
destination  immediately  following  the  initiation  of  the  command.  While  fire  fighting 
appliances  in  real  life  are  obviously  not  as  quick  responding,  having  the  fire  fighting 
appliances  travel  at  a  slower  rate  would  have  affected  the  overall  mean  idle  time, 
which  was  being  used  as  an  indication  of  the  maximum  number  of  fire  trucks  one 
could  cognitively  manage.  Therefore,  full  realism  in  this  aspect  of  fire  fighting 
appliance  behaviour  was  sacrificed  for  more  precise  and  controlled  manipulation  of 
the  number  of  appliances  one  could  cognitively  manage.  Pilot  testing  indicated  this 
compromise  was  readily  accepted  and  adapted  to  by  research  participants. 

For  the  experimental  trials,  one  scenario  was  created  and  then  horizontal, 
vertical,  and  vertical-horizontal  reflections  of  the  original  scenario  were  used  to  create 
four  scenarios  that  were  equivalent  in  landscape  features,  fire  and  wind  behaviour, 
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and  difficulty,  but  not  recognised  as  such  by  the  participants.  Two  sets  of 
experimental  trials  were  needed,  as  the  administration  of  the  SAGAT  immediately 
post  trial  was  considered  likely  to  interfere  with  the  administration  of  self-report 
questionnaires  (and  vice  versa).  The  first  two  experimental  trials  were  used  to 
administer  the  objective  SA  assessment.  The  second  two  experimental  trials  were 
used  to  administer  the  Individual  Trial  Subjective  Experience  Questionnaire  (see 
Appendix  C)  and  the  After  All  Trials  Questionnaire  (see  Appendix  D),  and  to  collect 
perfonnance  score  data.  The  four  experimental  trials  were  counterbalanced  in  terms  of 
appliance  availability  condition  and  landscape  orientation.  The  method  of 
counterbalancing  can  be  seen  in  Table  1. 


Table  1 

Counterbalancing  of  appliance  availability  condition  and  landscape  orientation 
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Design 

The  present  study  used  a  within-subjects,  repeated  measures  design  to  examine 
decision-making  performance,  SA  and  mental  workload  in  both  the  MANAGEABLE 
and  EXCESS  conditions.  Decision-making  performance  was  the  primary  dependant 
variable,  which  was  operationalised  as  the  performance  score  in  terms  of  value  of 
landscape  assets  saved  in  each  NFC  trial.  Other  dependent  variables  were  SA,  which 
was  measured  objectively  by  the  SAGAT  and  subjectively  via  self-report,  and  mental 
workload,  which  was  also  measured  via  self-report. 
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Measures 

Demographics 

A  background  information  questionnaire  was  used  to  gather  general 
background  information  about  participants,  including  their  age,  gender,  computer 
usage,  skills  and  confidence,  and  decision-making  experience  in  DDM  situations  (see 
appendix  A). 

DDM  Performance 

A  performance  score  was  automatically  recorded  by  NFC  after  each  trial, 
which  indicated  the  value  of  landscape  assets  (as  a  percentage)  remaining  unburnt  at 
the  end  of  a  trial  (Omodei,  Wearing,  McLennan  and  Hansen,  2001).  Such 
perfonnance  scores  served  as  the  primary  measure  of  the  participants  DDM  skill. 

For  analyses  of  the  extent  to  which  each  individuals’  score  on  the  EXCESS 
condition  trial  differed  from  their  score  on  the  corresponding  MANAGEABLE 
condition  trial,  residualised  change  scores  instead  of  raw  change  scores  were 
calculated  and  used.  Residualised  change  scores  have  the  advantage  over  raw  change 
scores  in  that  they  remove  any  change  due  to  regression  to  the  mean  in  correlated 
variables  (Tabachnick  &  Fidell,  2001). 

Situation  Awareness 

As  an  objective  assessment  of  SA,  a  shortened  version  of  Endsley’s  (1995b) 
Situation  Awareness  Global  Assessment  Technique  (SAGAT)  was  used.  In  the  first 
two  experimental  trials  comprising  MANAGEABLE  and  EXCESS  conditions,  the 
simulation  was  paused  at  an  unknown  time  to  participants  and  only  the  bare  landscape 
(i.e.,  minus  fire  and  fire  effects)  was  re-presented  on  the  computer  screen.  On  a 
transparent  acetate  sheet  attached  to  the  screen,  participants  were  then  asked  to  draw 
the  location  of  the  perimeter  of  the  fire  that  was  threatening  the  most  valuable 
landscape. 

This  item  regarding  awareness  of  the  most  threatening  fire  was  judged  to  be  a 
measure  of  global  SA.  Specifically,  participants  had  to  perceive  the  wind  strength  and 
direction  in  addition  to  the  location  of  fires  (level  1  SA),  before  comprehending  this 
infonnation  (level  2  SA)  to  decide  which  fire  would  be  threatening  the  most  valuable 
landscape  (i.e.,  houses)  in  the  near  future  (level  3  SA).  In  the  SAGAT  questionnaire, 
the  participants  also  had  to  mark  the  location  where  the  next  fire  was  due  to  break  out, 
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which  required  memory  of  the  location  of  a  fire  warning  that  had  been  presented 
immediately  prior  to  the  pause  in  the  simulation.  In  addition,  participants  had  to  mark 
the  strength  and  direction  of  the  wind  immediately  prior  to  the  pause  (see  appendix 
B).  Unfortunately  however,  the  first  fire  perimeter  item  took  the  full  20  seconds  to 
complete  for  most  participants,  so  the  data  obtained  from  additional  assessments  were 
not  used.  Specifically,  memory  decay  would  have  occurred  prior  to  the  completion  of 
the  additional  assessments,  so  they  would  not  have  been  reliable.  Subsequently,  the 
fire  perimeter  item  was  solely  used  as  the  global  measure  of  SA  for  all  analyses. 

The  drawings  on  the  transparent  acetate  sheet  were  eventually  compared  to  the 
real  scenario,  and  an  objective  score  of  global  SA  was  calculated  according  to  the 
following  scoring  code:  participants  received  a  score  out  of  9  for  awareness  of  the 
most  threatening  fire  (i.e.,  1  =  circled  an  area  that  was  not  particularly  close  to  any 
fire,  9  =  precisely  circled  the  entire  most  threatening  fire). 

The  pausing  of  the  simulation  occurred  at  3:40  mins  for  all  participants, 
although  the  timing  of  the  pause  was  unknown  to  the  participants.  This  was  an 
appropriate  time  that  was  sufficiently  cognitively  demanding,  whereby  there  were 
numerous  large  and  small  fires  that  had  broken  out.  Certainly,  the  number  and 
severity  of  fires  at  this  time  varied  among  participants  according  to  their  previous 
commands  and  skill  level.  However,  the  coding  system  used  for  the  marking  of  the 
SAGAT  was  explicitly  designed  to  take  these  differences  into  account,  thus  ensuring 
the  measure  of  SA  was  independent  of  decision  making  perfonnance. 

The  participant’s  subjective  rating  of  their  SA  was  also  measured,  using 
several  self-report  items  in  the  Individual  Trial  Subjective  Experience  Questionnaire 
completed  after  the  final  two  experimental  trials  (see  appendix  C).  Participants 
responded  to  questions  such  as:  ‘How  would  you  describe  your  overall  awareness  of 
what  was  going  on  during  the  last  Fire  Chief  trial?’  on  a  9-point  Likert- type  scale,  1  = 
very  low  awareness,  9  =  very  high  awareness. 

Subjective  Mental  Workload 

The  level  of  mental  workload  experienced  by  participants  in  each  condition 
was  measured  via  self-report  in  the  Individual  Trial  Subjective  Experience 
Questionnaire.  Specifically,  participants  responded  to  the  question  “How  much  mental 
effort  did  you  expend  overall”  on  a  9-point  Likert-type  scale  ( 1  =  no  mental  effort,  9  = 
a  great  deal  of  mental  effort),  and  this  item  was  used  as  the  main  measure  of  mental 
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workload.  Additionally,  sub-components  of  mental  effort  expenditure  were  assessed 
in  both  conditions,  whereby  participants  indicated  on  a  9-point  Likert-type  scale  (1  = 
no  mental  effort,  9  =  a  great  deal  of  mental  effort)  the  amount  of  mental  effort  they 
expended  on  (i)  scanning  the  environment  for  changes,  (ii)  deciding  what  action  to 
take  next,  (iii)  implementing  the  actions,  (iv)  controlling  emotional  reactions  and  (v) 
checking  the  right  approach  was  being  used  (see  Appendix  C).  As  numerous 
participants  verbalised  confusion  about  the  meaning  of  items  assessing  and  comparing 
the  amount  of  mental  effort  expended  controlling  emotions,  these  items  were 
excluded  from  analyses.  It  should  be  noted  that  responses  to  these  items  assessing 
mental  effort  expended  controlling  emotions  were  not  required  for  interpretation  of 
the  results  testing  the  main  research  hypothesis. 

Direct  comparisons  of  the  level  of  overall  mental  workload  experienced  and  of 
the  sub-components  of  mental  effort  expenditure  between  conditions  were  also 
obtained  in  the  After  All  Trials  Questionnaire  completed  at  the  end  testing  (see 
Appendix  D).  For  example,  participant  one  responded  to  questions  on  a  9-point 
Likert- type  scale;  1  =  more  mental  effort  overall  when  I  had  MANAGEABLE 
appliances,  5  =  the  same,  and  9  =  more  mental  effort  overall  when  I  had  EXCESS 
appliances  (note  the  order  of  presentation  of  these  anchors  corresponded  to  the  order 
in  which  the  trial  conditions  were  administered  to  each  participant).  It  is  important  to 
note  that  the  items  assessing  mental  effort  expended  overall  and  the  extent  to  which 
participants  had  a  heavy  mental  workload  were  also  used  as  a  check  that  the  NFC 
trials  were  sufficiently  cognitively  demanding  (see  Appendix  C).  Specifically,  a  score 
above  5  on  the  9-point  Likert-type  scale,  indicated  that  a  greater  than  moderate  level 
was  necessary  for  the  NFC  trials  to  be  considered  cognitively  demanding. 

Strategic  Thought  Allocated  to  Appliance  Usage 

The  items  assessing  the  amount  of  mental  effort  that  participants  expended  on 
checking  the  right  approach  was  being  used  and  deciding  what  action  to  take  next 
were  also  specifically  used  as  a  measure  of  the  strategic  thought  participants  allocated 
to  appliance  usage.  The  item  assessing  mental  effort  scanning  the  enviromnent  for 
changes  was  also  specifically  used  as  a  measure  of  mental  effort  allocated  to  situation 
assessment  and  SA  maintaining  procedures,  so  this  item  was  not  considered  as  a 
measure  of  strategic  thought.  Note  that  the  item  assessing  mental  effort  expended 
implementing  actions  was  not  considered  as  a  measure  of  strategic  thought,  because 
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this  item  was  merely  measuring  the  mental  effort  allocated  to  the  physical  action  of 
appliances,  not  their  strategic  placement.  For  example,  participants  with  slow 
psychomotor  skill  would  expend  a  high  amount  of  mental  effort  actually  moving  the 
appliances  and  clicking  on  them  to  take  action  and  vice-versa. 

Other  Subjective  Experiences  in  Experimental  Conditions 

Differences  in  participants’  feelings  of  confidence  and  perceived  perfonnance 
between  the  MANAGEABLE  and  EXCESS  conditions  were  assessed  by  self-report 
items  using  a  9-point  Likert-type  scale  on  the  After  All  Trials  Questionnaire  (see 
Appendix  D).  Self-perceived  performance  in  both  MANAGEABLE  and  EXCESS 
conditions  was  also  assessed,  using  a  9-point  Likert- type  scale  on  the  Individual  Trial 
Subjective  Experience  Questionnaire  (see  Appendix  C).  Also  included  in  this 
questionnaire  was  an  item  asking  participants  “How  determined  were  you  to  perform 
as  well  as  you  could  on  the  last  Fire  Chief  trial”,  whereby  participants  had  to  report 
above  5  on  the  9-point  Likert- type  scale  (1=  not  detennined  at  all,  5  =  moderately 
determined,  9  =  very  determined)  to  be  included  in  the  analyses  (see  Appendix  C). 

Participant  Awareness  of  Compulsion  to  Use  All  Appliances 

In  all  past  studies  using  NFC  to  study  the  possibility  of  an  overutilisation  of 
resources  bias,  the  item  on  the  After  All  Trials  Questionnaire  “To  what  extent  did  you 
feel  compelled  to  use  all  the  fire  trucks  at  your  disposal?”  has  been  used  as  a  measure 
of  participant’s  compulsion  to  use  all  available  appliances.  As  has  been  mentioned, 
the  overutilisation  bias  may  well  operate  on  a  relatively  unconscious  level,  so  decision 
makers  may  not  always  be  consciously  aware  of  their  compulsion  to  use  all  available 
resources  (Omodei  et  al.,  2003).  Therefore,  the  self-report  measure  of  compulsion 
may  not  be  reliable.  Certainly,  there  is  strong  evidence  in  the  literature  on  NFC  that 
people  generally  feel  compelled  to  use  all  available  resources  provided  to  them. 
Therefore,  this  item  may  be  better  used  as  a  measure  of  the  extent  to  which 
participants  are  aware  of  such  a  compulsion.  Hence,  it  was  included  in  the  present 
study  for  this  reason. 

Note  that  participant’s  awareness  of  the  extent  of  compulsion  to  use  all 
appliances  was  not  assessed  after  each  appliance  availability  condition  separately,  so 
as  not  to  introduce  demand  characteristics  by  revealing  the  primary  aim  of  the  study 
before  both  trial  conditions  had  been  administered. 
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Reliability  of  Measures 

The  only  potential  measure  of  reliability  of  items  used  after  each  resource 
level  condition  would  be  to  correlate  responses  on  the  same  item  across  these  two 
experimental  conditions  (effectively  giving  lower-bound  estimates,  as  there  may  well 
be  valid  differences  in  these  experiences  across  individuals  and  trials).  Considering 
the  small  sample  size  used  in  the  present  study,  such  lower-bound  estimates  of 
reliability  would  themselves  have  been  of  low  reliability.  Realistically,  there  was  no 
method  for  providing  direct  evidence  for  reliability  of  the  measures  adopted  in  the 
present  study,  except  for  SA  (see  below).  Fortunately  however,  past  studies 
investigating  the  overutilisation  bias  using  NFC  have  reported  high  reliability  of  all 
such  measures,  and  hence  their  use  in  the  present  study  was  warranted  (Omodei, 
Wearing,  McLennan  &  Hansen,  2001).  Particularly  worthy  of  mention,  high 
reliability  of  performance  scores  as  a  measure  of  DDM  skill  have  been  found,  with 
correlations  of  .7  or  greater  being  consistently  reported  (Bozovic,  2004;  Omodei, 
Wearing,  McLennan  &  Hansen). 

The  inter-rater  reliability  of  the  accuracy  of  the  recalled  fire  perimeter  item  as 
a  measure  of  global  SA  was  found  to  be  satifactory.  Specifically,  participants’ 
responses  on  the  SAGAT  questionnaire  were  marked  by  two  researchers  using  the 
same  SAGAT  marking  code.  Pearson  correlations  between  both  researchers’  ratings 
of  SAGAT  scores  presented  inter-rater  reliability  of  r  =  .98 ,p<  .01  for  the 
MANAGEABLE  condition  and  r  =  .81  ,p<  .01  for  the  EXCESS  condition. 

Procedure 

Each  participant  was  individually  tested.  After  reading  an  information 
statement,  the  participants  completed  the  written  informed  consent  fonn  and  the 
background  questionnaire.  Participants  were  then  trained  in  the  use  of  the  NFC 
program  through  the  presentation  of  a  demonstration  trial.  To  ensure  all  participants 
received  the  same  instructions,  a  scripted  fire  chief  protocol  was  read  to  each 
participant  explaining  the  features  of  the  program,  the  aim  of  the  simulation  and  the 
details  for  operating  the  simulated  fire  fighting  appliances  (see  appendix  E). 

The  participants  then  completed  the  first  practice  trial,  where  they  had  access 
to  4  fire  fighting  appliances.  Pilot  testing  revealed  that  even  the  least  skilled 
participants  could  handle  4  appliances  quite  well,  so  this  seemed  an  appropriate 
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number  of  appliances  to  make  available  for  the  first  trial.  Prior  to  completing  the 
second  practice  trial,  appropriate  adjustments  to  the  number  of  appliances  available 
for  each  participant  were  made  upon  consideration  of  the  mean  idle  time  statistic  of 
trial  one.  Specifically,  pilot  testing  on  five  participants  revealed  that  if  participants 
appeared  to  be  just  managing  all  appliances  available  to  them,  this  was  associated 
with  a  mean  idle  time  of  approximately  70%.  While  seemingly  high,  this  value  takes 
into  account  the  time  participants  spent  pausing  to  strategise  and  to  implement 
individual  actions.  Therefore,  one  fire  fighting  appliance  additional  to  the  number 
available  in  the  previous  trial  was  provided  if  the  mean  idle  time  of  the  previous  trial 
was  below  70%.  Otherwise,  4  fire  trucks  were  made  available  for  the  second  practice 
trial.  It  was  assumed  that  participants  would  get  better  in  NFC  with  practice,  which  in 
fact  was  the  case. 

In  the  second  practice  trial,  the  trial  was  paused  in  order  to  provide  practice  in 
completing  the  SAGAT  questionnaire.  After  the  SAGAT  was  completed,  the  trial  was 
resumed.  Following  this,  the  remaining  three  practice  trials  were  completed,  which 
were  of  progressively  increasing  difficulty.  After  each  practice  trial,  participants  had  a 
break,  during  which  time  the  experimenter  noted  the  mean  idle  time  and  made 
appropriate  changes  to  the  number  of  appliances  available  in  the  next  practice  trial.  In 
the  case  where  the  mean  idle  time  of  the  previous  trial  was  above  70%,  one  appliance 
was  subtracted  from  the  number  available  in  the  previous  trial.  By  the  end  of  the  fifth 
practice  trial,  the  number  of  appliances  that  gave  a  mean  idle  time  closest  to  70%  was 
recorded  to  be  the  maximum  number  of  appliances  that  the  particular  participant 
could  cognitively  manage  (see  appendix  F).  By  this  stage,  it  was  expected  that 
participants  had  reached  stability  in  the  maximum  number  of  appliances  they  could 
manage. 

After  completion  of  the  last  practice  trial,  participants  were  given  a  45  minute 
break.  Upon  return  from  their  break,  participants  completed  a  refresher  trial  before 
completing  the  first  two  experimental  trials.  In  both  of  these  experimental  trials,  the 
simulation  was  paused  at  the  predetermined  time  of  3:40  mins  and  the  SAGAT 
questionnaire  was  administered  before  the  trial  resumed.  After  a  10  minute  break, 
participants  then  completed  the  last  two  experimental  trials,  and  the  Individual  Trial 
Subjective  Experience  Questionnaire  was  administered  after  each  of  these  trials. 
Finally,  the  After  All  Trials  Questionnaire  was  completed.  Participants  were  thanked 
for  their  time,  paid,  and  allowed  to  leave. 
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Results 

Overview 

The  results  of  preliminary  analyses  are  presented  in  the  first  section,  which 
comprise  (i)  a  check  that  all  participants  had  sufficient  determination  levels  and 
understanding  of  the  NFC  task,  (ii)  the  relationships  of  the  demographic  variables  to 
the  main  dependent  variables,  namely  decision  making  performance,  SA  and  mental 
workload,  (iii)  a  check  of  skewness  and  kurtosis  values  for  decision  making 
performance  scores,  SAGAT  scores  and  all  subjective  experience  variables,  and  (iv)  a 
check  of  the  adequacy  of  the  NFC  task  to  study  DDM.  The  results  of  analyses  that 
address  the  main  study  hypotheses  are  presented  in  the  second  section.  Finally,  the 
results  of  supplementary  analyses  are  presented,  intended  to  assist  with  the 
interpretation  of  the  main  results. 

Note  that  considering  the  small  sample  size  used  in  this  study,  all  correlation 
results,  both  in  preliminary  and  supplementary  analyses  must  be  interpreted  with 
caution.  Nonetheless,  the  results  of  the  overall  pattern  of  such  correlations  are 
considered  in  the  interpretation  of  the  main  results,  for  which  there  was  adequate 
statistical  power. 


Preliminary  Analyses 
Exclusion  Criteria 

All  participants  reported  determination  levels  above  the  5  cut-off  value. 
However,  one  participant  who  had  limited  English  vocabulary  and  limited 
comprehension  of  the  English  language  was  excluded  from  all  analyses,  because  of 
communication  difficulties.  It  appeared  he  did  not  understand  the  requirements  of  the 
NFC  tasks,  further  supported  by  observation  of  his  consistently  poor  performance 
scores  in  all  NFC  trials.  The  mean  determination  score  of  remaining  participants  was 
7.85  (SD  =  0.99)  (on  a  9-point  Likert-type  scale). 
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Demographics 


The  means  and  standard  deviations  of  all  the  demographic  variables  for  the 
remaining  15  participants  (5  male  and  10  female)  can  be  seen  in  TablTable  2 
Means  and  standard  deviations  of  all  demographic  variables 


M 

SD 

Demographic  Variable 

Participant  age 

21.47 

4.05 

Days  a  week  using  computer 

5.33 

2.35 

Rating  of  computer  skills 

6.33 

2.16 

How  comfortable  moving  mouse 

8.27 

.80 

How  often  play  computer  games 

4.60 

2.82 

Rating  of  coping  ability  under  pressure 

6.33 

.90 

Experience  with  decision  making 

under  time  pressure 

5.13 

1.55 

Note :  N  =  15,  Responses  were  given  on  a  9-point  Likert-type  scale,  except  for  “days  per  week  using 
computer”  and  “participant  age”. 


As  can  be  seen  in  Table  2,  it  appears  that  participants  overall  were  quite 
skilled  and  comfortable  computer  users  and  had  a  reasonable  amount  of  experience 
with  decision  making  in  time  pressured  situations.  Pearson  correlations  were 
calculated  between  each  demographic  variable  and  the  main  study  dependent 
variables.  Table  3  contains  the  results  of  such  correlations  with  performance  score  in 
each  appliance  availability  condition.  There  were  no  significant  correlations  of  the 
demographic  variables  with  SA  or  mental  workload. 
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Table  3 

Correlations  between  all  demographic  variables  and  decision  making  performance 
scores 


Performance  Score  Difference  in  Performance 

_ MANAGEABLE  EXCESS  MANAGEABLE-EXCESS 

Demographic  Variable 


Participant  age 

-.07 

.16 

-.30 

Days  a  week  using 

computer 

.27 

.19 

.14 

Rating  of  computer 

skills 

.10 

.50 

-.52 

How  comfortable 

moving  mouse 

.23 

.32 

-.10 

How  often  play 

computer  games 

.75** 

71** 

.12 

Rating  of  coping  ability 

under  pressure 

.01 

.26 

-.32 

Experience  with 

decision  making 

under  time  pressure 

.22 

.24 

-.00 

Note :  **p<  .01  (2-tailed),  N  =  15. 

As  can  be  seen  in  Table  3,  how  often  participants  played  computer  games 
significantly  correlated  with  their  decision  making  performance  in  both  the 
MANAGEABLE  and  the  EXCESS  conditions.  There  were  no  other  substantial 
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correlations  between  the  demographic  variables  and  performance  or  variation  in 
perfonnance  between  the  two  conditions. 

Independent  samples  /-tests  were  performed  to  test  if  gender  had  a  significant 
affect  on  performance.  Results  indicated  that  males  performed  slightly  better  in  the 
MANAGEABLE  condition  than  females  (Males:  M  =  80.19,  SD  =  13.67;  Females:  M 
=  67.30,  SD  =  14.17),  yet  this  difference  was  not  significant,  t(  1 3)  =  1.68,/?>.05 
(two-tailed),  d  =  .93.  Interestingly,  males  performed  significantly  better  in  the 
EXCESS  condition  compared  to  females  (Males:  M  =  84.48,  SD  =  8.20;  Females:  M 
=  69.68,  SD  =  13.87),  t(13)  =  2.18,/K.05  (two-tailed),  d  =  1.21.  Note  that  the  sample 
size  for  between-groups  comparisons  was  very  low. 

Check  for  Skewness  and  Kurtosis 

Skewness  and  kurtosis  values  for  NFC  performance  scores,  SAGAT  scores 
and  all  subjective  experience  variables  were  less  than  3  SE  units  and  therefore 
assessed  as  normal  (Tabachnick  &  Fidell,  2001).  The  raw  performance  scores 
achieved  by  individuals  in  the  experimental  trials  can  be  seen  in  Appendix  F.  The 
means  and  standard  deviations  of  performance  and  SAGAT  scores  can  be  seen  in 
Table  4. 


Table  4 

Means  and  standard  deviations  of  decision  making  performance  and  SAGAT  scores 


M 

SD 

Min 

Max 

n 

Performance  Scores 

MANAGEABLE 

71.60 

14.90 

51.13 

96.41 

15 

EXCESS 

74.61 

13.97 

46.71 

98.95 

15 

SAGAT  Scores 

MANAGEABLE 

6.86 

1.80 

3.00 

8.00 

EXCESS 

6.36 

1.87 

3.00 

9.00 

14 

Note :  Performance  scores  were  out  of  100,  SAGAT  scores  were  given  out  of  9. 
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As  can  be  seen  in  Table  4,  the  range  of  performance  scores  in  experimental 
trials  was  quite  large,  which  suggests  that  the  method  of  making  each  house  in  a  trial 
at  risk  of  being  burnt  was  effective  for  distinguishing  poor  from  efficient  decision 
makers.  The  range  of  SAGAT  scores  was  also  quite  large,  which  suggests  that 
participants  did  differ  in  regards  to  their  SA. 

The  baseline  percentage  value  was  approximately  25.71%  for  decision  making 
perfonnance  in  both  experimental  trial  conditions.  This  number  represents  the  average 
overall  value  of  landscape  assets  remaining  unburnt  as  a  percentage  at  the  end  of  an 
experimental  trial  if  no  actions  were  taken  at  all  (Omodei,  Wearing,  McLennan  & 
Hansen,  2001).  The  counterbalancing  of  landscape  orientation  across  appliance 
availability  condition  ensured  that  any  remaining  slight  differences  in  baseline  values 
between  landscape  orientations  were  controlled  for  experimentally. 

Check  of  Adequacy  of  NFC  Task  to  Study  DDM 

Table  5  shows  the  means  and  standard  deviations  of  participants’  responses  to 
the  mental  workload  and  mental  effort  items  on  the  Individual  Trial  Subjective 
Experience  Questionnaire. 
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Table  5 

Means,  standard  deviations  and  one-sample  t-test  results  of  core  questions  relating  to 
participant  mental  workload  and  mental  effort  in  MANAGEABLE  and  EXCESS 
conditions 


MANAGEABLE 

EXCESS 

One-sample 

One-sample 

t-test 

t-test 

M  SD  (two-tailed) 

M  SD 

(two-tailed) 

Mental  Workload 


Measure 
Heavy  workload 


6.83  1.36  t(  14)  =  5.23,  6.67 

p  =  .00 


1.54  t(14)  =  4.18, 

p  =  .00 


Mental  effort 
overall 


6.60  1.45  t(14)  =  4.26,  6.40  1.30  t(14)  =  4.18, 

p  =  .00  p=  .00 


Difficulty 

managing 

mental 

workload 


4.77  1.82  *(14)  =  -.50,  4.53  2.03  t(  14)  = -.89, 

p>.  05  P~>- 05 


Note:  N  =  15,  Responses  were  given  on  a  9-point  Likert-type  scale,  where  5  represented  a  moderate 
level  of  mental  workload.  One-sample  t-test  was  performed  against  a  test  value  of  5,  which  indicated  a 
moderate  level. 

As  can  be  seen  in  Table  5,  mean  levels  of  reported  workload  and  mental  effort 
were  significantly  greater  than  a  moderate  level  in  both  conditions.  Therefore,  NFC 
was  deemed  to  be  sufficiently  demanding  for  studying  DDM.  However,  mean  levels 
of  difficulty  managing  mental  workload  were  not  significantly  greater  than  a  moderate 
level  in  either  condition. 
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Paired  samples  /-tests  were  also  performed,  which  revealed  no  significant 
differences  in  workload,  difficulty  in  managing  mental  workload  and  mental  effort 
expended  overall  items  between  MANAGEABLE  and  EXCESS  conditions,  /(  14)  = 
.37,  p>.05  (two-tailed),  d=. 20,  /(  14)  =  .56,  p>.05  (two-tailed),  d=. 30,  /(  14)  =  .46, 
p>.05  (two-tailed),  d  =.25,  respectively.  Table  6  contains  the  means  and  standard 
deviations  for  two  items  on  the  Comparison  of  Trials  Questionnaire,  which  required 
participants  to  rate  in  which  appliance  availability  condition  they  had  a  heavier  mental 
workload  and  expended  more  mental  effort  overall. 

Table  6 

Means,  standard  deviations  and  one-sample  t-test  results  of  “ heavier  mental 
workload ”  and  “ more  mental  effort  expended  overall’’  items  on  the  After  All  Trials 
Questionnaire 


M 

SD 

One-sample  /-test 

(two-tailed) 

Heavier  mental  workload 

5.13 

2.62 

t(14)  =  .20, 

p>.05 

More  mental  effort  overall 

5.33 

2.29 

t(  14)  =  .56, 

p>.05 

Note:  N  =  15,  Responses  were  given  on  a  9-point  Likert-type  scale:  1  =  heavier  mental  workload/  more 
mental  effort  overall  when  I  had  MANAGEABLE  fire  trucks;  5  =  the  same,  and  9  =  heavier  mental 
workload/  more  mental  effort  overall  when  I  had  EXCESS  fire  trucks.  One-sample  t- test  was 
performed  against  a  test  value  of  5,  which  participants  would  have  circled  if  they  felt  no  difference 
between  conditions. 

As  can  be  seen  in  Table  6,  participants  did  not  experience  a  heavier  mental 
workload  or  expend  more  mental  effort  overall  in  any  appliance  availability  condition, 
This  means  that  mental  effort  expended  could  not  account  for  any  of  the  variance  in 
perfonnance  scores  across  appliance  availability  conditions. 
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Main  Research  Analyses 
Main  Hypotheses 

1)  Comparison  of  decision  making  performance  scores  between  MANAGEABLE 
and  EXCESS  conditions 

It  was  expected  that  participants  would  perform  better  in  the  MANAGEABLE 
condition  compared  to  the  EXCESS  condition.  A  paired  samples  /-test  was  performed 
on  decision  making  perfonnance,  which  revealed  no  statistically  significant  difference 
between  the  MANAGEABLE  condition  ( M  =  71.60,  SD  =  14.90)  and  the  EXCESS 
condition  (M  =  74.61,  SD  =  13.97),  t(  14)  =  -1.08,  p>.05  (two-tailed),  d  =  .58. 

2)  Comparison  of  mental  workload  between  MANAGEABLE  and  EXCESS 
conditions 

It  was  expected  that  participants  would  experience  a  lower  degree  of  mental 
workload  in  the  MANAGEABLE  condition  compared  to  the  EXCESS  condition.  A 
paired  samples  /-test  was  perfonned  on  the  item  regarding  mental  effort  expended 
overall,  which  revealed  no  significant  difference  between  the  MANAGEABLE 
condition  (M  =  6.60,  SD  =  1.45)  and  the  EXCESS  condition  (M  =  6.40,  SD  =  1.30), 
t(  14)  =  ,46,/?>.05  (two-tailed),  d  =  .25. 

3)  Comparison  of  situation  awareness  between  MANAGEABLE  and  EXCESS 
conditions 

It  was  expected  that  participants  would  have  better  SA  in  the  MANAGEABLE 
condition  compared  to  the  EXCESS  condition.  A  paired  samples  /-test  was  performed 
on  global  SAGAT  scores,  which  revealed  no  significant  difference  between  the 
MANAGEABLE  condition  (M  =  6.86,  SD  =  1.80)  and  the  EXCESS  condition  (M  = 
6.36,  SD  =  1.87),  t(  1 3)  =  .63,/?>.05  (two-tailed),  d  =  .35. 
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Supplementary  Analyses 

Correlations  Between  Decision  Making  Performance  and  Situation  Awareness 
Pearson  correlations  between  decision  making  perfonnance  and  global 
SAGAT  scores  were  conducted,  the  results  of  which  can  be  seen  in  Table  7. 


Table  7 

Means  and  standard  deviations  of  decision  making  performance  and  SAGAT  scores  in 
MANAGEABLE  and  EXCESS  conditions,  and  the  correlations  between  these  scores 


M 

SD 

n 

r 

MANAGEABLE 

SAGAT  score 

6.86 

1.79 

14 

.24 

Performance  score 

71.60 

14.90 

15 

EXCESS 

SAGAT  score 

6.36 

1.87 

14 

.34 

Performance  score 

74.61 

13.97 

15 

AVERAGE  across 

conditions 

SAGAT  score 

6.61 

1.10 

15 

Performance  score 

73.11 

13.39 

14 

.46 

Note:  SAGAT  score  is  out  of  9,  Performance  score  is  out  of  100. 

As  can  be  seen  in  Table  7,  there  were  no  significant  correlations  between 
perfonnance  scores  and  SAGAT  scores.  However,  the  conelations  were  approaching 
significance,  especially  for  the  EXCESS  conditions  and  averaged  across  conditions. 

Correlations  Between  Situation  Awareness  and  Mental  Workload 

Pearson  correlations  between  global  SA  and  mental  workload  were  conducted, 
the  results  to  which  can  be  seen  in  Table  8. 
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Table  8 


Correlations  among  SAGAT scores  and  mental  workload  measures  in 
MANAGEABLE  and  EXCESS  conditions,  and  averaged  across  conditions 


MANAGEABLE 

SAGAT  score 

EXCESS 

Average 

Mental  Workload  Measure 

Heavy  workload 

.16 

-.47* 

-.23 

Mental  effort  overall 

.06 

-.57* 

-.37 

Difficulty  managing 

-.53* 

.00 

-.42 

mental  workload 

Note:  The  average  of  SAGAT  scores  across  the  two  conditions  was  used  for  the  correlation  with  the 
average  of  each  mental  workload  measure  across  conditions,  N  =  15  for  mental  workload  measures,  N 
=  14  for  SAGAT  scores  *p<.05  (1 -tailed). 

As  can  be  seen  in  Table  8,  the  general  pattern  of  correlations  was  in  the 
expected  negative  direction.  A  significant  relationship  was  found  between  the  extent 
to  which  participants  felt  they  had  a  heavy  mental  workload  and  their  SAGAT  score 
in  the  EXCESS  condition,  r  =  -.47.  Additionally,  a  significant  relationship  was  found 
between  the  participants’  experienced  mental  effort  overall  and  their  SAGAT  score  in 
the  EXCESS  condition,  r  =  -.57.  Meanwhile,  a  significant  relationship  was  found 
between  participants’  reports  of  their  difficulty  managing  mental  workload  and  their 
SAGAT  score  in  the  MANAGEABLE  condition,  r  =  -.53. 

Correlations  Between  Decision  Making  Performance  and  Mental  Workload 

Pearson  correlations  between  mental  workload  measures  and  decision  making 
perfonnance  were  conducted,  the  results  of  which  can  be  seen  in  Table  9. 
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Table  9 

Correlations  among  decision  making  performance  scores  and  mental  workload 
measures  in  each  condition  and  averaged  across  conditions 


Performance  score 

MANAGEABLE  EXCESS  Average 


Mental  Workload  Measure 


Heavy  workload 

.06 

-.01 

.15 

Mental  effort  overall 

.07 

-.34 

-.05 

Difficulty  managing 

-.10 

-.25 

-.19 

mental  workload 

Note :  The  average  of  each  mental  workload  item  across  the  MANAGEABLE  and  EXCESS  conditions 
was  correlated  with  average  performance,  N  =  15. 

As  can  be  seen  in  Table  9,  none  of  the  mental  workload  measures  correlated 
with  decision  making  performance  in  either  condition  or  across  conditions. 

Self-perceived  Performance  Across  Appliance  Availability  Conditions 

Participants  were  asked  to  subjectively  rate  their  performance  on  a  9-point 
Likert-type  scale  for  both  the  MANAGEABLE  and  EXCESS  conditions  on  the 
Individual  Trial  Subjective  Experience  Questionnaire.  A  paired  samples  /-test  was 
perfonned,  which  revealed  no  significant  difference  in  perceived  performance 
between  the  MANAGEABLE  condition  (M  =  6.67,  SD  =  1.1 1,  n  =  15)  and  the 
EXCESS  condition  (M  =  6.00,  SD  =  1.36,  n  =  15),  t(14)=  1.67,  p>. 05  (two-tailed),  d 
=  .89.  Interestingly,  the  correlation  between  self-reported  performance  in 
MANAGEABLE  and  EXCESS  conditions  was  low  (r  =  .24)  despite  the  much  higher 
correlation  between  objective  perfonnance  in  both  conditions  (r  =  .72).  As  there  may 
well  be  strong  actual  differences  in  self-perceived  performance  across  conditions,  this 
low  correlation  may  perhaps  best  be  interpreted  as  indicating  such  actual  differences. 

On  the  After  All  Trials  Questionnaire,  participants  were  asked  to  subjectively 
rate  which  condition  they  felt  they  had  performed  best  on  by  responding  on  a  9-point 
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Likert-type  scale  (i.e.,  1  =  performed  best  when  I  had  MANAGEABLE  appliances,  5 
=  performed  equally  well  in  both  conditions,  9  =  perfonned  best  when  I  had  EXCESS 
appliances).  A  one  sample  /-test  performed  against  a  test  value  of  5  revealed  that 
participants  generally  did  not  feel  they  performed  better  in  any  particular  condition  (M 
=  5.13,  SD  =  2.42,  n  =  15),  *(14)  =  .21,  p>.05  (2-tailed). 

The  correlation  between  participant’s  self-perceptions  of  perfonnance  and 
actual  perfonnance  scores  in  the  MANAGEABLE  condition  was  slightly 
insignificant,  r  =  .38.  /;>.05.  Participant’s  self-perceptions  of  performance  did 
correlate  significantly  with  perfonnance  scores  in  the  EXCESS  condition,  r  =  .59, 
p<. 05.  Interpreted  simply,  the  results  suggest  that  participants  seemed  to  have  quite 
good  awareness  of  their  level  of  performance,  specifically  in  the  EXCESS  condition, 
but  were  insensitive  to  differences  in  their  performance  across  the  two  conditions. 

Pearson  correlations  were  performed  between  participants’  self-perceptions  of 
their  perfonnance,  mental  workload,  and  global  SA,  the  results  of  which  can  be  seen 
in  Table  10. 


Table  10 

Correlations  among  self-perceived  performance,  mental  workload,  and  SAGAT  scores 

Self-perceived  perfonnance 
MANAGEABLE  EXCESS  Average 
Mental  Workload  Measure - 


Heavy  workload 

-.11 

-.31 

-.11 

Mental  effort  overall 

-.18 

-.48* 

-.08 

Difficulty  managing 

mental  workload 

-.16 

-.41 

-.21 

SAGAT  score 

Awareness  of  threatening 

fire  item  .31  74**  .75** 


Note :  *p<.05  (1-tailed),  **p<.01  (1-tailed),  N  =  15  for  correlations  with  mental  workload  and  self- 
reported  SA,  N  =  14  for  correlations  with  SAGAT. 
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As  can  be  seen  in  Table  10,  there  were  few  statistically  significant 
relationships  between  self-perceived  performance  and  mental  workload.  However,  a 
significant  negative  correlation  was  found  between  mental  effort  expended  overall 
and  self-perceived  performance  in  the  EXCESS  condition.  Specifically,  as 
participants’  mental  effort  overall  increased,  their  self-perceived  performance 
decreased  in  the  EXCESS  condition. 

SAGAT  scores  were  significantly  correlated  to  self-perceived  perfonnance  in 
the  EXCESS  condition  and  averaged  across  conditions,  but  not  in  the 
MANAGEABLE  condition.  In  examining  the  overall  pattern  of  these  correlations,  it 
can  be  noted  that  stronger  associations  were  obtained  in  the  EXCESS  condition  than 
in  the  MANAGEABLE  condition. 

Self-perceived  Situation  Awareness  Across  Appliance  Availability  Conditions 

Pearson  correlations  were  performed  between  participants  self-perceived  SA, 
decision  making  performance  scores  and  SAGAT  scores,  the  results  of  which  can  be 
seen  in  Table  11. 

Table  1 1 

Correlations  between  self-reported  SA,  SAGAT  scores  and  decision  making 
performance  scores 

Global  SAGAT  Score  Performance  Score 

MANAGEABLE  EXCESS  Ave.  MANAGEABLE  EXCESS  Ave 

Self  reported 
Situation  Awareness 

Overall  Awareness  .42  -.07  .26  -.06  -.04  .07 


Note :  The  average  of  Global  SAGAT  scores  and  Performance  scores  across  MANAGEABLE  and 
EXCESS  conditions  was  correlated  with  average  self-reported  SA,  N  =  15  for  correlations  with  self- 
reported  SA  and  performance  score,  N  =  14  for  correlations  with  SAGAT  scores. 

As  can  be  seen  in  Table  11,  there  were  no  statistically  significant  relationships 
between  self-reported  overall  SA,  SAGAT  scores  and  performance  scores.  Consistent 
with  Brozovic  (2004),  the  lack  of  correlation  between  self-reported  SA  and  SAGAT 
scores  is  not  surprising,  considering  that  participants  had  only  their  perceptions  of 
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what  was  happening  in  the  task,  not  what  was  realistically  happening  (Brozovic, 
2004).  Likewise  therefore,  the  fact  the  participants’  general  poor  knowledge  of  their 
SA  was  not  related  to  their  actual  decision  making  performance  is  also  not  surprising. 
Additionally,  self-perceived  performance  was  not  significantly  related  to  self- 
perceived  SA. 

Sub-components  of  Mental  Effort  Across  Appliance  Availability  Conditions 
Means  and  standard  deviations  of  comparisons  of  mental  effort  sub¬ 
components  across  the  MANAGEABLE  and  EXCESS  conditions  can  be  seen  in 
Table  12. 


Table  12 

Means,  standard  deviations  and  one-sample  t-test  results  of  comparisons  of  mental 
effort  sub-components  across  the  MANAGEABLE  and  EXCESS  conditions 


M 

SD 

One-sample  t-test 

(two-tailed) 

Mental  Effort  Component 

Checking  approach 

4.00 

1.77 

t(14)  =  -2.19,/?  <  .05 

Deciding  on  next  action 

3.47 

1.46 

*(14)  =  -4.08,/?  <  .01 

Implementing  actions 

4.67 

1.68 

t(14)  =  -.77,p  >  .05 

Scanning  the  environment 

4.13 

2.00 

t(14)  =  -1.68,/?  >  .05 

Note:  N  =  15,  Responses  to  these  items  were  given  on  a  9-point  Likert-type  scale:  1  =  more  mental 
effort  when  I  had  MANAGEABLE  fire  trucks,  5  =  the  same,  and  9  =  more  mental  effort  when  I  had 
EXCESS  fire  trucks.  One-sample  t-test  was  performed  against  a  test  value  of  5,  which  participants 
would  have  circled  if  they  felt  no  difference  between  conditions. 

As  can  be  seen  in  Table  12,  participants  expended  more  mental  effort  checking 
that  they  were  using  the  right  approach  and  deciding  which  action  to  take  next  in  the 
MANAGEABLE  condition.  Therefore,  participants  allocated  more  strategic  thought 
to  appliance  usage  in  the  MANAGEABLE  condition  compared  to  the  EXCESS 
condition.  Also,  mental  effort  expended  implementing  actions  was  not  significantly 
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different  between  the  conditions.  Finally,  the  finding  of  no  significant  difference  in 
mental  effort  scanning  the  environment  for  changes  between  conditions  suggests  that 
the  same  mental  effort  was  dedicated  to  situation  assessment  and  SA  maintaining 
procedures  in  both  conditions. 

Pearson  correlations  were  performed  between  participants’  overall  mental 
effort  expended  and  specific  components  that  such  mental  effort  was  expended  on  in 
NFC.  Table  13  contains  the  results  of  such  correlations. 

Table  13 

Correlations  between  the  overall  mental  effort  item  and  specific  mental  effort  sub¬ 
components  in  the  MANAGEABLE  and  EXCESS  conditions 


Overall  Mental  Effort 
MANAGEABLE  EXCESS 


Mental  Effort  Component 

Scanning  the  environment 

.36 

Deciding  on  next  action 

.14 

.68** 

Implementing  actions 

.15 

.34 

Checking  approach 

.35 

.61** 

Note:  *p<.05  (1-tailed),  **p<.01  (1-tailed),  N=  15 

Table  13  shows  that  the  less  mental  effort  overall  in  the  EXCESS  condition 
was  accompanied  by  less  effort  scanning  the  environment  for  changes,  deciding  on 
what  action  to  take  next  and  checking  the  right  approach  was  being  used.  In 
examining  the  overall  pattern  of  these  correlations,  it  can  be  noted  that  stronger 
associations  were  obtained  in  the  EXCESS  condition  than  in  the  MANAGEABLE 
condition. 

Attention  to  All  Parts  of  the  Scenario 

Global  SAGAT  score  was  positively  related  to  the  item  assessing  participants’ 
attention  to  all  parts  of  the  scenario  in  both  the  MANAGEABLE  condition  (r  =  .47) 
and  the  EXCESS  condition  (r  =  .46),  although  the  latter  did  not  quite  reach 
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significance  level.  No  other  correlations  between  this  attention  item  and  other 
variables  were  found. 

Participant  Awareness  of  Compulsion  to  Use  All  Appliances 

Participants  indicated  on  a  9-point  Likert-type  scale  ( 1  =  not  compelled  to  use 
all  appliances,  9  =  very  compelled  to  use  all  appliances)  that  they  were  aware  of  a 
strong  compulsion  to  use  all  available  appliances  across  MANAGEABLE  and 
EXCESS  conditions  overall  (M  =  7.53,  SD  =  1.77).  Furthermore,  a  one-sample  t-test 
was  performed  against  a  test  value  of  5,  which  found  that  the  mean  level  of 
experienced  compulsion  was  significantly  greater  than  a  moderate  awareness  level  of 
compulsion  t(  14)  =  5.55,  p  =  .00  (two-tailed). 

Maximum  Number  of  Appliances  That  Could  Be  Cognitively  Managed 

For  further  insight,  the  maximum  number  of  appliances  manageable  variable 
(as  determined  by  the  experimenter  on  the  basis  of  the  mean  idle  time)  was  correlated 
with  all  demographic  variables,  perfonnance  scores  and  global  SAGAT  scores,  the 
results  to  which  can  be  seen  in  Table  14. 
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Table  14 

Correlations  between  the  maximum  number  of  appliances  manageable  variable  and 
performance  scores,  global  SAGAT scores  and  all  demographic  variables 


Maximum  Number  of  AnDliances  Manaaeable 

Demoaranhic  Variables 

Participant  age 

-.21 

Days  a  week  using 

computer 

.40 

Rating  of  computer 

skills 

.28 

How  comfortable 

moving  mouse 

.46 

How  often  play 

computer  games 

73** 

Rating  of  coping  ability 

under  pressure 

.15 

Experience  with 

decision  making 

under  time  pressure 

.10 

Performance  Scores 

MANAGEABLE  condition 

77** 

EXCESS  condition 

.69** 

Global  SAGAT  Scores 

MANAGEABLE  condition 

.17 

EXCESS  condition 

.18 

Note:  *p<.05  (1-tailed),  **p<.01  (1-tailed),  N  =  15. 
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As  can  be  seen  in  Table  14,  how  often  participants  played  computer  games 
was  significantly  correlated  to  the  maximum  number  of  appliances  they  could 
cognitively  manage.  Furthennore,  an  independent  samples  t-test  revealed  that  males 
(M  =  6.20,  SD  =  1.30)  could  cognitively  manage  more  fire  fighting  appliances  than 
females  (M=  5.00,  SD  =  0.67),  /(13)  =  2.40,/><.05  (2-tailed?),  d  =  1.33.  The  large 
range  in  the  maximum  amount  of  appliances  that  could  be  managed  does,  in  fact, 
reflect  genuine  differences  in  cognitive  capacities.  As  expected  therefore,  the  higher 
the  number  of  appliances  a  participant  could  cognitively  manage,  the  higher  their 
perfonnance  in  both  the  MANAGEABLE  and  EXCESS  conditions. 

Participants  Who  Performed  Exceptionally  Better  In  a  Particular  Condition 

Residualised  change  scores  to  identify  participants  as  showing  an  unduly 
positive  or  negative  change  in  perfonnance  from  MANAGEABLE  to  EXCESS 
conditions  were  calculated.  Following  this,  the  top  and  bottom  five  participants 
according  to  residualised  change  scores  were  selected  and  compared.  Specifically, 
several  independent  sample  /-tests  were  conducted  to  compare  participants  who 
perfonned  particularly  poorly  in  the  EXCESS  condition  relative  to  the 
MANAGEABLE  condition,  with  participants  who  performed  particularly  better  in  the 
EXCESS  condition  relative  to  the  MANAGEABLE  condition,  on  all  main  study 
variables  and  self-report  items. 

As  expected,  there  was  a  significant  difference  in  perfonnance  in  the  EXCESS 
condition  between  participants  relatively  better  in  MANAGEABLE  (M  =  62.91,  SD  = 
13.55)  and  participants  relatively  better  in  EXCESS  (M  =  84.23,  SD  =  9.07),  /(8)  =  - 
2.92.  p<S)5  (1-tailed),  d  =  2.06.  Interestingly,  it  was  also  found  that  participants  who 
were  relatively  better  in  MANAGEABLE  expended  significantly  more  mental  effort 
deciding  what  action  to  take  next  in  the  EXCESS  condition  (M  =  7.70,  SD  =  .97) 
compared  to  participants  relatively  better  in  EXCESS  (M  =  4.80,  SD  =  1.79),  /(8)  = 
3.18,  /;<.05  (2-tailed),  d  =  2.25.  Additionally,  it  was  found  that  participants  who  were 
relatively  better  in  MANAGEABLE  expended  significantly  more  mental  effort 
checking  that  they  were  using  the  right  approach  in  the  EXCESS  condition  (M  =  6.00, 
SD  =  0.00)  compared  to  participants  relatively  better  in  EXCESS  (M  =  3.80,  SD  = 
1.79),  /(8)  =  2.75,/?<.05  (2-tailed),  d  =  1.94.  Therefore,  participants  relatively  better 
in  MANAGEABLE  allocated  more  strategic  thought  to  appliance  usage  in  the 
EXCESS  conditions  than  participants  relatively  better  in  EXCESS. 
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It  would  be  expected  that  the  participants  relatively  better  in  MANAGEABLE 
would  have  had  a  higher  mental  workload  and  struggled  handling  this  mental 
workload  in  the  EXCESS  condition.  In  accordance,  participants  relatively  better  in 
MANAGEABLE  expended  significantly  more  mental  effort  overall  in  the  EXCESS 
condition  (M=  7.20,  SD  =  1.30)  compared  to  participants  relatively  better  in 
EXCESS  (M=  5.40,  SD  =  1.14),  t{ 8)  =  2.32, ^<.05  (1-tailed),  d  =  1.64.  Also, 
participants  relatively  better  in  MANAGEABLE  had  significantly  more  difficulty 
managing  their  mental  workload  in  the  EXCESS  condition  (M=  5.40,  SD  =  1.95) 
compared  to  participants  relatively  better  in  EXCESS  (M  =  3.00,  SD  =  2.00),  /(8)  = 
1.92,/?<.05  (1-tailed),  d  =  1.35.  No  other  significant  relationships  were  found  for  any 
other  variable. 


Discussion 

Overview 

The  first  part  of  this  section  comprises  a  discussion  of  the  results  of  the 
preliminary  analyses.  This  includes  a  discussion  of  the  effect  of  demographic 
variables  and  of  the  adequacy  of  NFC  to  study  DDM.  Following  consideration  of  the 
results  of  the  preliminary  and  supplementary  analyses,  the  results  of  the  main  analyses 
are  discussed  in  terms  of  the  specific  hypotheses.  Finally,  practical  implications  of 
the  findings  and  suggestions  for  future  research  are  discussed.  As  has  already  been 
mentioned,  the  correlations  must  be  interpreted  cautiously  because  of  the  small 
sample  size.  Nonetheless,  such  results  give  some  insights  into  the  interpretation  of  the 
main  findings. 

Preliminary  Analyses 
Demographic  Variables 

Interestingly,  the  more  frequently  participants  played  computer  games,  the 
higher  their  performance  score  was  in  both  appliance  availability  conditions.  Future 
research  using  a  larger  sample  is  needed  to  investigate  whether  participants  who  play 
computer  games  more  frequently  are  more  efficient  decision  makers  in  computer 
simulated  microworlds  such  as  NFC,  or  just  more  practised  at  the  psychomotor  skills 
involved  in  managing  the  interface. 
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Also  interesting  is  the  relationship  between  gender  and  decision  making 
perfonnance.  Specifically,  males  could  cognitively  manage  significantly  more 
appliances  than  females.  Furthennore,  males  performed  significantly  better  than 
females  in  the  EXCESS  condition,  and  slightly  but  not  significantly  better  in  the 
MANAGEABLE  condition.  It  is  important  to  note  however,  only  five  males  were 
tested  in  the  present  study  compared  to  ten  females.  Again,  future  research  using  a 
larger  sample  is  needed  to  re-investigate  these  findings.  Specifically,  such  future 
research  should  explore  whether  males  in  general  have  greater  overall  cognitive 
capacity  and  processing  efficiency  than  females  with  respect  to  complex  dynamic 
tasks,  or  are  just  more  practiced  at  the  psychomotor  skills  involved  in  managing  the 
NFC  interface.  Furthermore,  future  research  should  explore  the  possibility  that  males 
may  have  more  effective  task  management  strategies  to  deal  with  an  over-abundance 
of  resources. 

Adequacy  of  the  NFC  Task  for  Studying  DDM 

Overall,  participants  reported  experiencing  a  heavy  workload  and  expending  a 
large  amount  of  mental  effort  in  both  appliance  availability  conditions.  Therefore,  the 
NFC  trials  were  sufficiently  cognitively  demanding  to  test  the  research  hypotheses  in 
the  present  study.  Interestingly,  participants  generally  did  not  report  having  significant 
difficulty  managing  their  mental  workload  in  either  condition.  This  seems  to  suggest 
that  the  participants  generally  had  effective  task  management  strategies  to  deal  with 
the  task  load  in  both  conditions.  However,  as  it  will  be  discussed  later,  there  were  in 
fact  individual  differences  in  the  ability  to  appropriately  adapt  to  the  extra  resources  in 
the  EXCESS  condition. 

Main  Research  Analyses 

As  it  will  become  apparent,  the  level  of  strategic  thought  allocated  to 
appliance  usage  by  participants  in  both  conditions  merits  attention  in  the  discussion  of 
the  decision  making  performance  results.  Hence,  these  items  are  discussed  in  the 
following  section  addressing  the  results  of  the  first  hypothesis  instead  of  in  the  section 
addressing  the  final  hypothesis  regarding  mental  workload. 
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Comparisons  of  Decision  Making  Performance  and  Strategic  Thought  Between 
MANAGEABLE  and  EXCESS  Conditions 

The  hypothesis  that  decision  making  performance  would  be  better  in  the 
MANAGEABLE  condition  compared  to  the  EXCESS  condition  was  not  supported.  In 
fact,  there  was  no  significant  difference  in  performance  between  appliance  availability 
conditions.  Therefore,  one  might  infer  that  participants  were  able  to  deal  effectively 
with  the  extra  appliances  in  the  EXCESS  condition  without  overloading  their  working 
memory  (Brozovic,  2004).  However,  if  this  was  the  case,  their  decision  making 
perfonnance  should  have  been  better  in  the  EXCESS  condition  compared  to  the 
MANAGEABLE  condition  because  more  appliances  offer  more  fire  fighting  power 
(Omodei  &  Wearing,  2004).  This  was  not  found,  so  one  might  alternatively  infer  that 
performance  was  impaired  to  at  least  some  extent  in  the  EXCESS  condition.  Of 
particular  relevance  are  the  results  of  further  analyses  suggesting  that  in  fact,  the 
perfonnance  of  only  some  participants  was  impaired  in  the  EXCESS  condition  (i.e., 
participants  relatively  better  in  MANAGEABLE).  Importantly,  the  results  of  further 
analyses  also  suggest  that  the  more  strategic  thought  allocated  to  appliance  usage  in 
the  EXCESS  condition,  the  more  decision  making  performance  was  impaired  in  that 
condition.  This  finding  is  discussed  in  more  detail  below. 

Participants  who  perfonned  relatively  better  in  the  MANAGEABLE  condition 
allocated  more  strategic  thought  to  appliance  usage  in  the  EXCESS  condition  than 
participants  who  performed  relatively  better  in  EXCESS.  Therefore,  participants 
relatively  better  in  MANAGEABLE  appeared  to  be  trying  to  utilize  the  over¬ 
abundance  of  appliances  in  a  more  strategic  manner,  which  led  to  their  working 
memory  and  attention  capacities  being  exceeded.  In  support  of  this,  participants 
relatively  better  in  MANAGEABLE  had  a  higher  mental  workload  and  had  more 
difficulty  managing  their  mental  workload  in  the  EXCESS  condition  than  participants 
relatively  better  in  EXCESS,  a  finding  that  will  be  discussed  in  further  detail  later. 

In  light  of  these  results,  the  sacrifice  of  strategic  thought  to  take  advantage  of 
the  extra  appliances  in  the  EXCESS  condition  seemed  appropriate  for  achieving 
control  of  the  simulated  spreading  fires  in  NFC.  Conversely,  the  use  of  appliances  in  a 
strategic  manner  was  perhaps  advantageous  in  the  MANAGEABLE  condition. 
Providing  support  for  this,  participants  overall  allocated  more  strategic  thought  to 
appliance  usage  in  the  MANAGEABLE  condition  than  in  the  EXCESS  condition.  It 
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is  important  to  note  that  this  finding  also  suggests  that  even  participants  relatively 
better  in  MANAGEABLE  perhaps  reduced  strategic  thought  associated  with 
appliance  use,  at  least  to  some  extent,  in  the  EXCESS  condition. 

Perhaps  the  attempt  to  use  the  over-abundance  of  appliances  meant  that  some 
decrement  in  strategic  thought  was  inevitable.  Specifically,  participants  would  have 
had  fewer  cognitive  resources  available  for  strategic  thought.  However,  perhaps 
participants  relatively  better  in  MANAGEABLE  did  not  sacrifice  strategic  thought  in 
the  EXCESS  condition  to  the  same  extent  that  participants  relatively  better  in 
EXCESS  did.  Therefore,  it  may  be  said  there  are  perhaps  individual  differences  in  the 
extent  to  which  strategic  thought  is  sacrificed  in  the  presence  of  an  over-abundance  of 
resources.  Future  research  using  a  larger  sample  is  needed  to  investigate  this 
proposition. 

The  general  finding  that  participants  sacrificed  strategic  thought  more  in  the 
EXCESS  condition  provides  some  evidence  that  the  participants  were  perhaps 
compelled  to  use  the  over-abundance  of  appliances.  Important  to  note  however,  the 
finding  of  no-difference  in  self-perceived  perfonnance  between  the  conditions  is 
contrary  to  the  proposal  that  all  people  believe  “more  is  better”  with  regard  to 
infonnation  and  other  decision  resources.  Rather,  only  some  individuals  might  hold 
this  belief.  Also  important  to  note,  perceptions  of  performance  generally  correlated 
with  actual  performance  and  therefore,  the  low  correlation  in  such  perceptions  across 
conditions  suggest  participant  insensitivity  to  the  effect  of  the  resource  availability 
manipulation  on  their  perfonnance.  Such  results  may  have  important  implications  for 
the  current  theoretical  understanding  regarding  the  overutilisation  bias.  Specifically, 
there  may  be  individual  differences  in  the  extent  to  which  such  a  compulsion  to  use  all 
available  appliances  is  experienced,  which  was  not  assessed  in  the  present  study. 
Therefore,  future  research  investigating  the  overutilisation  bias  should  perhaps 
incorporate  a  measure  of  the  extent  of  compulsion  to  use  all  resources  in  each 
resource  availability  condition.  Suggestions  for  ways  to  measure  this  compulsion  will 
be  discussed  in  further  detail  later. 

Comparison  of  Mental  Workload  Between  MANAGEABLE  and  EXCESS 
Conditions 

The  hypothesis  that  mental  workload  would  be  lower  in  the  MANAGEABLE 
condition  compared  to  the  EXCESS  condition  was  not  supported.  Mental  effort 
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expended  overall,  difficulty  managing  mental  workload  and  the  extent  to  which 
participants  felt  they  had  a  heavy  mental  workload  were  not  significantly  different 
between  the  two  conditions.  Importantly,  participants  reported  high  motivation  and 
reported  expending  a  high  amount  of  mental  effort  in  both  appliance  availability 
conditions.  Therefore,  there  may  have  been  a  ceiling  effect  for  the  mental  workload 
items,  whereby  the  self-reported  levels  for  these  items  could  not  get  any  higher  in  the 
EXCESS  condition  than  they  already  were  in  the  MANAGEABLE  condition 
(Brozovic,  2004).  Alternatively,  one  might  interpret  the  results  as  again  suggesting 
that  participants  were  able  to  deal  effectively  with  the  extra  appliances  in  the 
EXCESS  condition  without  overloading  their  working  memory.  However,  as  it  has 
already  been  argued,  there  were  in  fact  individual  differences  in  this  ability.  Likewise 
therefore,  there  were  perhaps  individual  differences  in  the  level  of  mental  workload 
experienced  across  conditions. 

Support  for  such  individual  differences  in  mental  workload  exist  in  the 
aforementioned  finding  that  participants  relatively  better  in  MANAGEABLE  had  a 
significantly  heavier  mental  workload  and  had  significantly  greater  difficulty 
managing  their  mental  workload  in  the  EXCESS  condition,  compared  to  participants 
relatively  better  in  EXCESS.  In  light  of  these  results,  it  may  also  be  that  the 
participants  relatively  better  in  MANAGEABLE  had  poorer  metacognitive  control 
and  thus  poorer  ability  to  self-monitor  and  self-regulate  the  state  of  their  working 
memory  in  the  EXCESS  condition  (Omodei,  Wearing,  McLennan,  Elliot  &  Clancy, 
2001;  Omodei  et  al.,  2003).  Therefore,  it  may  be  concluded  that  there  are  individual 
differences  in  the  ability  for  metacognitive  control,  and  for  self-monitoring  and  self¬ 
regulation  of  the  state  of  working  memory  in  the  presence  of  an  over-abundance  of 
resources. 

It  must  be  acknowledged  that  participants  who  were  relatively  better  in 
MANAGEABLE  still  only  reported  having  moderate  difficulty  managing  their  mental 
workload  in  the  EXCESS  condition.  However,  the  fact  that  these  participants  had 
such  poor  self-regulation  and  self-monitoring  of  the  state  of  their  working  memory 
meant  they  were  not  entirely  aware  of  the  extent  to  which  they  were  actually 
struggling  to  manage  their  mental  workload  in  the  EXCESS  condition.  As  previously 
noted,  such  participants  subsequently  tried  to  use  all  appliances  in  the  EXCESS 
condition  in  a  strategic  manner.  Future  research  is  needed  to  test  such  individual 
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differences  in  the  experience  and  management  of  high  mental  workload  in  DDM 
environments. 

Comparison  of  Situation  Awareness  Between  MANAGEABLE  and  EXCESS 
Conditions 

The  hypothesis  that  participants’  SA  would  be  better  in  the  MANAGEABLE 
condition  than  the  EXCESS  condition  was  not  supported.  Rather,  there  was  no 
significant  difference  in  SA  between  the  conditions.  Considering  that  participants 
generally  did  not  feel  cognitively  overloaded,  one  might  suggest  that  such  a  result  is 
understandable.  Specifically,  one  may  infer  that  participants  had  the  cognitive 
resources  available  to  maintain  a  similar  level  of  SA  in  the  EXCESS  condition  that 
they  had  achieved  in  the  MANAGEABLE  condition.  However,  this  inference  is 
invalidated  by  certain  results  obtained  from  further  analyses,  which  are  discussed 
below. 

Participants  relatively  better  in  MANAGEABLE  would  have  presumably  had 
less  available  cognitive  capacity  for  situation  assessment  and  thus  SA  maintaining 
procedures  than  participants  relatively  better  in  EXCESS.  This  is  because  participants 
relatively  better  in  MANAGEABLE  had  a  higher  mental  workload  and  greater 
difficulty  managing  their  workload  in  the  EXCESS  condition.  Nonetheless,  there  was 
no  significant  difference  in  SA  in  the  EXCESS  condition  between  participants 
relatively  better  in  MANAGEABLE  and  participants  relatively  better  in  EXCESS. 
Even  more  interesting  was  the  finding  that  there  was  no  significant  difference  in  SA 
between  these  participants  in  the  MANAGEABLE  condition.  It  may  be  the  case  that 
these  participants  simply  had  equal  ability  for  acquiring  and  maintaining  adequate  SA 
(Endsley  &  Bolstad,  1994).  Specifically,  the  participants  may  have  had  similar  spatial, 
perceptual,  attention  sharing  and  pattern  matching  skills  (Endsley  &  Bolstad). 
However,  in  the  present  study,  the  extent  of  such  skills  in  participants  can  only  be 
inferred  from  SA  scores.  Future  research  on  the  specific  factors  that  influence  one’s 
ability  to  acquire  and  maintain  SA  is  needed. 

Nonetheless,  participants  relatively  better  in  MANAGEABLE  and  participants 
relatively  better  in  EXCESS  must  have  dedicated  similar  amount  of  cognitive  capacity 
to  situation  assessment  and  thus  SA  maintaining  procedures  in  both  conditions.  In 
support  of  this,  self-reports  of  attention  to  all  parts  of  the  scenario  and  mental  effort 
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expended  scanning  the  environment  for  changes  was  not  different  in  either  condition 
between  these  participants. 

The  lack  of  correlation  between  SAGAT  scores  and  self-perceived  SA  merits 
attention.  In  effect,  this  suggests  that  self-reports  of  situation  awareness  are  unlikely 
to  provide  a  reliable  or  valid  indicator  of  such  awareness,  but  may  provide  useful 
infonnation  on  the  persons’  own  reactions  to  the  task  across  the  various  conditions. 
Hence,  future  studies  may  benefit  from  continuing  to  use  this  item. 

Also  interesting  to  note,  participants  who  had  a  heavier  mental  workload  and 
expended  more  mental  effort  overall  in  the  EXCESS  condition  achieved  a  lower 
SAGAT  score.  This  provides  small  yet  noteworthy  evidence  that  the  development  and 
maintenance  of  higher  order  S  A  may  be  hindered  by  heavy  mental  workload 
(Brozovic,  2004).  However,  despite  the  differences  in  mental  workload,  SA  and  self- 
reported  attention  to  all  parts  of  the  scenario  in  both  conditions  were  not  significantly 
different  between  participants  relatively  better  in  MANAGEABLE  and  participants 
relatively  better  in  EXCESS.  Therefore,  future  research  is  needed  to  investigate  the 
difference  in  SA  between  participants  who  do  and  do  not  make  appropriate  changes  in 
their  strategies  in  the  presence  of  more  resources  than  can  be  cognitively  managed. 
Specific  ways  of  assessing  the  nature  and  effectiveness  of  strategies  people  use  to  deal 
with  an  over-abundance  of  appliances  will  be  proposed  later. 

Also  interesting  was  the  finding  that  the  correlations  between  decision  making 
perfonnance  and  SA  were  positive  and  approaching  significance,  especially  for  the 
EXCESS  condition  and  averaged  across  conditions.  Again,  future  research  using  a 
larger  sample  is  needed  to  investigate  this  finding.  Specifically,  future  research  should 
explore  Endsley’s  (1995a)  proposition  that  SA  might  increase  the  probability  of  good 
decision  making  performance,  but  cannot  necessarily  guarantee  it. 

Potential  Limitations  of  the  Present  Study 

One  might  argue  that  a  limitation  of  the  present  study  was  the  measurement  of 
SA,  because  the  three  levels  of  SA  were  not  individually  assessed.  However,  as  it  has 
been  discussed,  items  included  on  the  SAGAT  to  measure  the  specific  levels  would 
not  have  been  reliable  because  of  memory  decay.  Fortunately,  the  fire  perimeter  item 
was  a  global  measure  of  SA,  which  required  LI,  L2  and  L3  SA.  Providing  small  but 
valuable  support  for  this  argument,  participants  who  reported  good  attention  to  all 
parts  of  the  scenario  rather  than  one  aspect  of  the  scenario  achieved  significantly 


46 


better  global  SAGAT  scores,  especially  in  the  MANAGEABLE  condition.  Certainly, 
the  idea  of  using  only  one  item  on  the  SAGAT  questionnaire,  which  essentially 
measured  global  SA,  was  appropriate  for  avoiding  the  issue  of  memory  decay 
typically  encountered  in  previous  studies  using  NFC. 

One  might  also  express  concern  about  the  reliance  on  the  mean  idle  time 
statistic  as  a  measure  of  the  maximum  amount  of  appliances  one  could  cognitively 
manage.  It  might  be  suggested  that  individual  differences  in  the  psychomotor  skill, 
errors  made  in  the  initiation  of  commands  and  in  the  strategies  used  to  deal  with 
appliances  could  affect  the  mean  idle  time  value.  However,  participants  with  low 
psychomotor  skill,  who  made  more  errors  or  who  were  slower  in  the  implementation 
of  commands  due  to  strategic  thought,  would  have  presumably  had  less  remaining 
cognitive  capacity  for  extra  appliances  than  other  participants.  Hence,  the  maximum 
amount  of  appliances  that  could  be  cognitively  managed  would  consistently  give  a 
mean  idle  time  of  70%,  regardless  of  skill  level  and  strategy  use. 

The  fact  that  the  NFC  trial  replays  only  show  the  appliances  responding  to  a 
specific  command  means  it  is  unclear  whether  noticeably  long  periods  of  time  of  no 
action  are  the  result  of  poor  participant  psychomotor  skill,  action  implementation 
error,  or  time  spent  by  the  participant  strategising  about  the  placement  of  appliances. 
Therefore,  individual  differences  in  the  extent  of  experienced  compulsion  to  use  all 
resources  and  in  the  extent  of  quantity-quality  trade-offs  could  not  be  reliably 
assessed  in  the  present  study.  Needless  to  say,  the  results  of  the  present  study  are  still 
valuable,  yet  future  insight  could  be  gained  into  the  nature  of  the  overutilisation  bias  if 
the  NFC  program  was  improved  to  incorporate  the  mouse  cursor  activity  in  trial 
replays. 

Practical  Implications  of  Findings 

Most  importantly,  the  present  study  has  contributed  a  unique  perspective  to  the 
research  already  conducted  on  the  nature  of  the  overutilisation  bias.  Perhaps  it  is  not 
sufficient  to  generalise  that  all  people  suffer  from  a  deeply  embedded  cognitive  bias 
of  a  tendency  to  overuse,  nor  is  it  appropriate  to  generalise  that  people  are  generally 
poor  decision  makers  when  there  is  an  over-abundance  of  resources.  In  fact,  it  has 
become  apparent  that  despite  human  processing  limitations,  some  people  can  take 
advantage  of  the  perfonnance  edge  offered  by  the  availability  of  an  abundance  of 


resources. 
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This  finding  has  important  practical  implications.  Previous  suggestions  of 
Omodei  and  colleagues  to  restrict  the  number  of  resources  available  to  decision 
makers  in  DDM  situations  may  not  be  relevant  for  all  individuals.  In  fact,  more 
resources  may  be  better  for  some  individuals,  but  only  if  they  are  able  to  adapt  their 
decision  making  strategies  to  take  advantage  of  the  over-abundance  of  resources. 
Therefore,  perhaps  the  focus  should  be  on  developing  training  programs  that  assist 
people  to  adapt  to  variations  in  resource  availability. 

The  finding  of  individual  differences  in  the  maximum  number  of  resources 
that  can  be  cognitively  managed  also  has  important  practical  implications.  Selecting 
emergency  management  personnel  who  can  cognitively  manage  a  large  number  of 
resources  to  be  leading  decision  makers  in  the  DDM  situations  would  be  appropriate 
for  minimising  the  potential  for  fatal  error  and  maximising  the  potential  for  control. 

Suggestions  for  Future  Research 

Much  in  the  same  way  as  in  the  present  study,  future  research  investigating  the 
possibility  of  an  overuse  of  resources  bias  in  NFC  generated  scenarios  ought  to 
control  for  individual  differences  in  the  maximum  number  of  resources  one  can 
cognitively  manage.  The  use  of  the  mean  idle  time  statistic  as  a  measure  of  the 
maximum  amount  of  appliances  one  can  cognitively  manage  was  reliable,  and 
warrants  use  in  future  research.  Also,  the  shortened  version  of  Endsley’s  (1995b) 
SAGAT  was  reliable,  and  warrants  use  in  future  studies  investigating  people’s  SA  in 
NFC. 

If  the  NFC  trial  relays  were  improved  to  show  mouse  cursor  activity,  perhaps 
future  researchers  could  use  NFC  trial  replays  to  assess  the  extent  to  which  people 
feel  compelled  to  use  an  over-abundance  of  resources.  From  this,  the  extent  to  which 
participants  observably  make  trade-offs  between  the  quantity  of  resources  used  and 
the  quality  of  strategic  thought  associated  with  such  resource  use  could  be  inferred. 

Despite  the  results  of  the  present  study,  perhaps  it  is  not  appropriate  to 
generalise  that  decreasing  strategic  thought  is  an  appropriate  strategy  for  every 
person.  Individual  differences  in  experience,  psychomotor  skill,  cognitive  capacity 
and  processing  efficiency  might  mean  there  are  differences  in  the  extent  to  which 
sacrificing  quality  of  strategic  thought  to  deal  with  the  over-abundance  of  appliances 
is  effective.  This  remains  an  avenue  worthy  of  exploration  in  future  studies. 


48 


It  is  interesting  to  note  that  the  need  for  cognitive  closure  may  be  a  predictor 
of  whether  the  sacrifice  or  maintenance  of  quality  of  strategic  thought  is  appropriate 
for  particular  individuals  when  there  is  an  over-abundance  of  resources  in  DDM 
environments.  Therefore,  future  studies  should  measure  people’s  need  for  cognitive 
closure,  which  is  manifested  through  several  different  aspects,  such  as  desire  for 
predictability,  preference  for  order  and  structure,  and  discomfort  with  ambiguity, 
decisiveness,  and  close-mindedness  (Webster  &  Kruglanski,  1994).  Specifically, 
people  with  low  need  for  cognitive  closure  are  more  rapid  decision  makers,  so  the 
sacrifice  of  quality  of  strategic  thought  to  take  advantage  of  an  over-abundance  of 
appliances  would  be  appropriate  for  them  (Joslyn  &  Hunt,  1998;  Webster  & 
Kruglanski,  1994).  Conversely,  people  with  a  high  need  for  cognitive  closure  would 
prefer  and  benefit  from  using  fewer  appliances  in  a  strategic,  orderly  and  structured 
manner.  A  measure  of  the  need  for  cognitive  closure  could  be  obtained  using  the  Need 
for  Cognitive  Closure  Scale  commonly  used  in  social  cognition  research,  which  was 
found  to  be  a  highly  reliable  and  valid  instrument  (see  Webster  &  Kruglanski,  1994, 
for  extensive  discussion  and  review  of  the  Need  for  Cognitive  Closure  Scale). 

Future  studies  should  measure  people’s  psychomotor  skill,  which  could  also 
be  considered  in  the  process  of  rating  the  effectiveness  of  quantity-quality  trade-offs 
made  by  participants.  Those  with  quick  psychomotor  skill  can  achieve  control  of 
spreading  fires  in  NFC  through  the  quick  implementation  of  actions,  so  sacrificing 
quality  of  strategic  thought  to  take  advantage  of  an  over-abundance  of  appliances 
would  be  an  appropriate  strategy  for  such  participants.  Conversely,  participants  with 
slower  psychomotor  skill  would  benefit  from  using  a  select  few  appliances  in  a 
strategic  manner  rather  than  trying  to  use  all  available  appliances.  A  measure  of 
participants’  psychomotor  skill  could  be  obtained  by  having  them  complete  a  simple 
NFC  trial,  where  they  are  instructed  to  complete  a  series  of  specific  commands  in  the 
quickest  possible  time. 
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Conclusion 

The  present  study  investigated  the  purported  phenomenon  of  the  human 
tendency  to  overuse  resources,  called  the  overutilisation  bias,  after  controlling  for 
individual  differences  in  the  number  of  resources  cognitively  manageable.  It  was 
generally  found  that,  in  the  presence  of  more  appliances  than  could  be  cognitively 
managed,  participants’  mental  workload,  SA  and  decision  making  perfonnance  were 
not  significantly  affected.  However,  these  results  also  suggest  that  participants 
generally  did  not  appear  to  be  able  to  take  advantage  of  the  performance  edge  offered 
by  the  availability  of  an  over-abundance  number  of  resources.  Of  major  interest, 
further  analyses  revealed  there  may  be  individual  differences  in  the  extent  of 
compulsion  experienced  to  use  an  over-abundance  of  appliances  when  made 
available.  Furthermore,  there  were  individual  differences  in  the  ability  to 
appropriately  adapt  decision  making  strategies  to  take  advantage  of  the 
overabundance  of  resources.  In  the  present  study,  allocating  less  strategic  thought  to 
appliance  usage  to  take  advantage  an  over-abundance  of  appliances  was  appropriate 
for  achieving  control  of  the  dynamic  situation  in  NFC. 

However,  it  was  acknowledged  there  may  in  fact  be  individual  differences  in 
the  extent  to  which  specific  quantity-quality  trade-offs  are  appropriate.  It  was 
suggested  that  one’s  need  for  cognitive  closure  and  one’s  psychomotor  skill  may 
predict  what  quantity-quality  trade-off  might  be  appropriate  in  the  presence  of  an 
over-abundance  of  decision  making  resources.  This  remains  to  be  determined  by 
future  research. 

It  is  concluded  that  individual  flexibility  in  the  quality  of  strategic  thought 
allocated  to  resource  usage  may  well  be  a  major  predictor  of  decision  making 
efficiency  in  dynamic  environments  such  as  those  encountered  by  fire  fighters. 
Hence,  individual  differences  in  strategic  flexibility  merit  investigation  in  future 
research  into  the  nature  of  decision  making  under  varying  levels  of  decision  resource 
availability. 
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Appendices 
APPENDIX  A 

BACKGROUND  INFORMATION  QUESTIONNAIRE 

Question  1 

a)  Please  circle  one  number  to  show:  Male  =  1  Female  =  2 

b)  Age  in  years  _ 

Question  2 

Computer  Experience:  (circle  one  number  per  question) 

a)  How  many  days  per  week,  on  average,  do  you  use  a  computer? 

0  1  2  3  4  5  6  7 


b)  In  general,  how  good  are  your  computer  skills? 


1 

2 

3 

4 

5 

6 

7 

8 

9 

Very  poor 

Average 

Advanced 

c)  How  comfortable  do  you  feel  manoeuvring  a  computer  mouse? 

1  2  3  4  5  6  7 

Very 

uncomfortable 

The  mouse  is 
ven> 

distracting  and 
difficult  to  use 


d)  How  often  do  you  play  computer  games? 

1234  5  6789 

Never  Sometimes  All  the 

time 


8  9 

Very 

comfortable 

I  don ’t  even 
think  about 
it 
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Question  3 


a)  How  well  do  you  cope  under  pressure,  when  decisions  need  to  be  made 
quickly  and  accurately? 

123456789 

Very  Average  Very  well 

poorly 


b)  Have  you  had  much  experience  making  decisions  under  time  pressure  and 
uncertainty  (ie  emergency,  stressful,  managerial  situations)? 


1 

2 

3 

4 

5 

6 

7 

8 

9 

No 

Some 

A  lot  of 

experience 

experience 

experience 

I  have  to 
make  these 
kinds  of 
decisions 
even’  day 

END  OF  QUESTIONNAIRE 
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APPENDIX  B 

SAGAT  QUESTIONNAIRE 

It  is  important  in  completing  this  task  that  you  provide  your  first  impressions  rather 
than  careful  consideration  as  to  whether  you  might  be  right  or  wrong.  That  is,  just  do 
as  well  as  you  can  without  focusing  on  whether  your  responses  are  right  or  wrong. 

Question  1 

On  the  clear  acetate  sheet  on  the  computer  screen; 

i.  Out  of  the  fires  that  are  still  burning,  use  the  black  pen  to  draw  the  fire  that  is 
threatening  the  most  valuable  landscape. 

ii.  Using  the  blue  pen,  mark  the  location  where  the  next  fire  is  due  to  break  out. 


Question  2 


Which  of  the  pictures  below  most  closely  matches  the  strength  of  the  current  wind? 


b) 

c) 

d) 

Current  Wind 

Current  Wind 

Current  Wind- 

Current  Wind  | 

§ 

§ 

@| 

©  1 

Question  3 


On  the  picture,  please  mark  the  direction  of  the  current  wind. 


Current  Wind 

© 


END  OF  QUESTIONNAIRE 
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APPENDIX  C 

INDIVIDUAL  TRIAL  SUBJECTIVE  EXPERIENCE  QUESTIONNAIRE 

The  following  questions  relate  to  your  experience  in  the  trial  you  have  just  completed. 
Please  indicate  your  answer  to  each  of  the  following  questions  by  circling  one  number 
on  the  scale  provided. 

Question  1 

a)  How  well  do  you  think  you  performed  in  terms  of  the  value  of  landscape  saved  in 
the  last  Fire  Chief  trial? 

1  2  3  4  5  6  7  8  9 

not  well  very  well 

at  all 


Question  2 

a)  How  determined  were  you  to  perfonn  as  well  as  you  could  on  the  last  Fire  Chief 
trial? 


1  2  3  4  5  6 

not  determined 
at  all 


7  8  9 

very 

determined 


b)  How  much  mental  effort  did  you  put  into  the  last  Fire  Chief  trial? 


1  2  3  4  5  6 

no  effort 


7  8  9 

a  great  deal 
of  effort 
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Question  3 


a)  To  what  extent  did  you  feel  that  you  had  a  heavy  workload  during  the  last  Fire 
Chief  trial? 


123456789 

very  light  very  heavy 

workload  workload 

b)  How  difficult  was  it  for  you  to  manage  the  mental  workload  required  of  you  in  the 
last  Fire  Chief  trial? 


123456789 

not  difficult  vety 

at  all  difficult 


c)  In  the  last  Fire  Chief  trial  how  much  mental  effort  did  you  expend  overall? 


1  2  3  4  5  6 

no  mental 
effort 


7  8  9 

a  great  deal  of 
mental  effort 


d)  In  the  last  Fire  Chief  trial  how  much  mental  effort  did  you  expend  scanning  the 
environment  for  changes  or  new  developments? 


1  2  3  4  5  6 

no  mental 
effort 


7  8  9 

a  great  deal  of 
mental  effort 


e)  In  the  last  Fire  Chief  trial  how  much  mental  effort  did  you  expend  deciding  what 
action  to  take  next? 


1  2  3  4  5  6 

no  mental 
effort 


7  8  9 

a  great  deal  of 
mental  effort 
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f)  In  the  last  Fire  Chief  trial  how  much  mental  effort  did  you  expend  on 
implementing  the  actions  you  had  decided  on? 


1  2  3  4  5  6 

no  mental 
effort 


7  8  9 

a  great  deal  of 
mental  effort 


g)  In  the  last  Fire  Chief  trial  how  much  mental  effort  did  you  expend  on  controlling 
your  emotional  reactions  to  the  situation? 


123456789 

no  mental  a  great  deal  of 

effort  mental  effort 


h)  In  the  last  Fire  Chief  trial  how  much  mental  effort  did  you  expend  on  checking  that 
you  were  using  the  right  approach  to  the  situation? 


123456789 

no  mental  a  great  deal  of 

effort  mental  effort 


Question  4 

a)  How  would  you  describe  your  overall  awareness  of  what  was  going  on 
during  the  last  Fire  Chief  trial? 


123456789 

very  low  very  high 

awareness  awareness 


b)  To  what  extent  were  you  aware  of  the  overall  distribution  of  fire  trucks  during  the 
last  Fire  Chief  trial? 


1  2  3  4  5  6 

not  aware 
at  all 


7  8  9 

very  much 
aware 
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c)  To  what  extent  were  you  aware  of  the  current  wind  conditions  during  the  last  Fire 
Chief  trial? 


1  2  3  4  5  6 

not  aware 
at  all 


7  8  9 

very  much 
aware 


d)  To  what  extent  were  you  aware  of  the  forecast  wind  conditions  during  the  last  Fire 
Chief  trial? 


1  2  3  4  5  6 

not  aware 
at  all 


7  8  9 

very  much 
aware 


e)  To  what  extent  were  you  aware  of  fire  warnings  during  the  last  Fire  Chief  trial? 


1  2  3  4  5  6 

not  aware 
at  all 


7  8  9 

very  much 
aware 


f)  How  well  do  you  feel  you  prioritised  the  different  fire  fronts  in  the  last  Fire  Chief 
trial? 


1  2  3  4  5  6 

not  well  at  all 


7  8  9 

very  well 


g)  In  the  last  Fire  Chief  trial,  to  what  extent  did  you  find  yourself  becoming  so 
involved  with  fighting  a  particular  fire,  that  you  didn’t  pay  attention  to  what  else 
was  happening  in  other  parts  of  the  screen? 


1  2  3  4  5 

very  involved  with  just 
one  fire,  did  not  pay 
atten  tion  to  other  parts 


6  7  8  9 

never  involved  in  just 
one  fire,  always  paid 
attention  to  other  parts 


60 


END  OF  QUESTIONNAIRE 
APPENDIX  D 

AFTER  ALL  TRIALS  QUESTIONNAIRE 


The  following  questions  relate  to  your  experiences  in  the  last  four  trials  you  have  just 
completed.  Please  indicate  your  answer  to  each  of  the  following  questions  by  circling 
one  number  on  the  scale  provided. 


Question  1 


Overall,  in  which  Fire  Chief  trials  do  you  think  you  performed  best  on? 


1  2 

3 

4  5  6 

7  8  9 

performed  best  when 

I  had  _  fire  trucks 

the  same 

performed  best  when  I 
had  _  fire  trucks 

Question  2 

a)  In  general,  to  what  extent  did  you  feel  compelled  to  use  all  the  fire  trucks  at  your 
disposal? 

1  2  3  4  5  6  7  8  9 

not  compelled  very  compelled 

at  all 


b)  To  what  extent  did  you  deliberately  keep  some  fire  trucks  idle  when  you  had 
fire  trucks  available? 

1  2  3  4  5  6  7  8  9 

never  always 


c)  To  what  extent  did  you  deliberately  keep  some  fire  trucks  idle  when  you  had 
_  fire  trucks  available? 

1  2  3  4  5  6  7  8  9 

never  always 
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Question  3 


a)  In  which  Fire  Chief  trials  did  you  feel  you  had  a  heavier  mental  workload? 


123456789 


heavier  mental  same 

workload  when  I 
had  _  fire  trucks 


heavier  mental 
workload  when  I 
had  _  fire  trucks 


b)  In  which  Fire  Chief  trials  did  you  feel  more  confident? 


/  2 

3 

4  5  6 

7 

8  9 

more  confident 

the  same 

more  confident 

when  I  had  _ 

when  I  had  _ 

fire  trucks 

fire  trucks 

c)  In  which  Fire  Chief  trials  did  you  expend  more  mental  effort  overall? 


123456789 

more  mental  effort  the  same  more  mental  effort 

when  I  had  _  fire  trucks  when  I  had  _  fire  trucks 


d)  In  which  Fire  Chief  trials  did  you  expend  more  mental  effort  on  scanning  the 
environment  for  changes  and  new  developments? 


1  2 

3 

4  5  6 

7 

8  9 

more  mental  effort 
on  scanning  when 

I  had  _  fire  trucks 

the  same 

more  mental  effort 
on  scanning  when 
I  had  _  fire  trucks 
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e)  In  which  Fire  Chief  trials  did  you  expend  more  mental  effort  on  deciding  what 
action  to  take  next? 


1  2 

3 

4  5  6 

7 

8  9 

more  mental  effort 

the  same 

more  mental  effort 

on  deciding  when 

on  deciding  when 

I  had  _  fire  trucks 

I  had  _  fire  trucks 

f)  In  which  Fire  Chief  trials  did  you  expend  more  mental  effort  on  implementing  the 
actions  you  had  decided  on? 


1  2 

more  mental  effort 
on  implementing 
I  had  _  fire  trucks 


4  5  6 

the  same 


8  9 

more  mental  effort 
on  implementing 
I  had  _  fire  trucks 


g)  In  which  Fire  Chief  trials  did  you  expend  more  mental  effort  on  controlling  your 
emotional  reactions  to  the  situation? 


123456789 


more  mental  effort  the  same 

controlling  emotions 
when  I  had  _  fire  trucks 


more  mental  effort 
controlling  emotions 
when  I  had  _  fire  trucks 


h)  In  which  Fire  Chief  trials  did  you  expend  more  mental  effort  on  checking  that  you 
were  using  the  best  approach  to  the  situation? 


12  3  4 


6  7  8  9 


more  mental  effort 
on  checking 

when  I  had  _  fire  trucks 


5 

the  same 


more  mental  effort 
on  checking 
when  I  had  _  fire  trucks 
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Question  4 


In  the  Fire  Chief  task,  the  situation  is  constantly  changing  (e.g.  new  fires  breaking  out, 
wind  direction  changing).  In  which  Fire  Chief  trial  did  you  feel  you  kept  better  track 
of  things? 


/  2 

3 

4  5  6 

7 

8  9 

kept  better  track 

the  same 

kept  better  track 

of  things  when  I 

of  things  when  I 

had  _  fire  trucks 

I  had  _  fire  trucks 

Question  5 


In  the  Fire  Chief  task,  it  is  usually  necessary  to  place  higher  priority  on  some  fires 
over  others.  In  which  Fire  Chief  trials  did  you  feel  you  set  better  priorities? 


12  3 

4  5  6 

7  8  9 

better  priorities 

the  same 

better  priorities 

when  I  had  _  fire  trucks 

when  I  had  _  fire  trucks 

Question  6 


Fire  Chief  scenarios  are  so  complex  and  fast-paced  that  mistakes  are  inevitable.  In 
which  Fire  Chief  trials  did  you  feel  you  made  more  mistakes? 


12  3 

4  5  6 

7 

8  9 

made  more  mistakes 
when  I  had  _  fire  trucks 

the  same 

made  more  mistakes 
I  had  _  fire  trucks 

END  OF  QUESTIONNAIRE 
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APPENDIX  E 


FIRE  CHIEF  PROTOCOL 


Mark  the  time  when  you  start  the  experiment  (important  if  participants  get  paid.) 

Give  out  and  collect  information  statement,  consent  forms  and  Demographic 
Questionnaire. 

Have  participant  sit  at  the  computer. 

Experimenter  sits  on  right  side  of  subject. 

Have  water  available  for  subject. 

Have  3  acetate  sheets  prepared. 

Load  the  demonstration  trial.  Do  NOT  begin  running  it  yet. 

Explain  that  you  will  be  reading  the  protocol  so  that  everyone  gets  the  same 
instructions. 

Ensure  you  are  following  and  entering  file  numbers  (those  in  the  brackets)  not  trial 
numbers  when  prompted  by  the  computer. 


Introduction  to  the  Task 

You  are  participating  in  a  study  of  complex  decision-making  that  makes  use  of  a  computer- 
simulated  fire-fighting  task.  The  simulation  task  we  are  using  is  called  Fire  Chief.  The  Fire 
Chief program  is  a  computer  recreation  of  a  real-world  fire-fighting  situation.  It  is  basically 
a  map  of  an  area  where  fires  break  out  and  your  job  as  the  Fire  Chief  is  to  extinguish  the 
fires. 


Over  the  next  couple  of  hours  you  will  be  trained  in  the  use  of  the  Fire  Chief  program.  First,  1 
will  introduce  you  to  the  computer  simulation  with  a  demonstration  and  then  I’ll  get  you  to 
complete  a  basic  scenario.  Then,  I’ll  give  you  a  chance  to  become  more  familiar  with  the 
program,  through  five  10  minute  practice  trials.  You  will  have  a  short  break  between  each 
practice  trial  whereby  I  will  ask  you  to  leave  the  room  so  I  can  make  necessary  reviews  and 
recordings.  At  the  end  of  the  last  practice  trial,  you  will  be  given  a  45  minute  break.  When 
you  return,  you  will  be  run  through  a  1 0  minute  refresher  trial.  Then  you  will  be  run  through 
two  more  trials.  During  the  course  of  these  trials,  I  will  ask  you  to  reproduce  parts  of  the  fire 
scenario  screen  by  drawing  them  onto  a  transparent  film.  Finally,  you  will  be  run  through  a 
further  two  10  minute  trials  and  the  computer  will  record  your  performance  on  all  these  trials. 
During  the  course  of  these  trials,  I'll  ask  you  to  fill  out  brief  questionnaires  about  your 
experiences  as  the  Fire  Chief. 

Now,  I  will  concentrate  on  showing  you  how  to  operate  the  computer  simulation  and  give  you 
a  chance  to  have  a  go  yourself.  At  first,  it  may  seem  as  though  there  is  a  lot  of  new 
information  for  you  to  take  in  but  try  and  remember.  The  long  explanations  will  usually  be 
followed  by  an  opportunity  for  you  to  have  a  go  on  the  computer  yourself.  Having  said  that; 
please  feel  free  to  ask  questions  at  any  stage  throughout  the  practice  sessions. 


Bring  the  demonstration  trial  up.  Go  through  this  slowly. 
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Explanation  of  Task  Features 


i)  Landscape 

What  you  can  see  on  your  screen  is  a  demonstration  map,  which  shows  the  different  types  of 
landscapes  and  fire  fighting  appliances. 

The  brown  houses  represent  residential  areas. 


All  the  green  areas  represent  bushland. 


[POINT  OUT  using  pen/curser] 


[POINT  OUT  using  pen/curser] 


All  the  blue  areas  represent  water  sources. 


[POINT  OUT  using  pen/curser] 


ii)  Fire  Trucks 

Fire  fighting  trucks  are  also  referred  to  as  appliances.  The  trucks  are  based  at  various  fire 
stations  in  the  trials. 

Fire  stations:  solid  red  squares 

Trucks:  light  blue  outlined  squares 

[POINT  OUT  using  pen/curser] 

Trucks  and  fire  stations  cannot  be  burnt. 


iii)  Fire  Fighting  Priorities 


The  task  is  designed  so  that,  just  as  in  real  life,  some  landscape  features  are  more  important 
than  others.  Residential  areas  are  the  most  important  areas  to  save  from  fire  destruction. 
Therefore,  bushland  is  less  important  than  residential  areas.  Remember  this  when  fighting  the 
fires.  While  there  may  be  a  lot  of  fires  spreading  in  a  trial  to  the  point  that  it  may  feel 
overwhelming,  you  can  still  achieve  a  high  performance  score  if  you  are  concentrating  on 
saving  the  houses. 


The  comparative  value  for  each  landscape  feature  is: 


Flousing:  100  points 
Bushland:  1  point 


Let’s  start  the  demonstration. 


START  demonstration  trial. 
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iv)  Fires 

Y ou  will  see  on  the  screen  that  a  fire  has  broken  out.  The  fires  are  represented  by  the  orange 
flames.  Once  landscape  is  burnt  it  turns  black  and  cannot  be  burnt  again. 


[POINT  OUT  using pen/curser] 


There  is  therefore  no  use  trying  to  save  landscape  that  has  already  burnt  out.  Fires  with  orange 
flames  are  the  ones  that  are  currently  burning  and  need  to  be  extinguished. 


v)  Simulation  Running  Time 


The  number  on  the  top  left-hand  side  of  the  screen  indicates  the  number  of  minutes  and 
seconds  that  have  passed  since  the  start  of  the  simulation. 

[POINT  OUT  using  pen/curser] 


vi)  Wind  Direction,  Wind  Strength  and  Wind  Forecast 

With  regard  to  the  demonstration  we  are  looking  at,  you  have  probably  noticed  that  the  fires 
are  spreading  more  quickly  in  the  SE  (say  wind  direction  that  is  appropriate  for  the  current 
trial)  direction  than  in  other  directions.  This  is  because  the  fires  spread  in  the  direction  of  the 
wind,  and  the  stronger  the  wind  the  faster  the  fire  spreads.  The  current  wind  details  are 
represented  by  the  wind  gauge  near  the  top  left-hand  side  of  your  screen. 


[POINT  OUT  using  pen/curser] 


The  size  of  the  blue  solid  circle  represents  the  strength  of  the  wind.  A  large  circle 
indicates  strong  wind  while  a  small  circle  indicates  a  light  breeze.  The  outer  circle  is  the 

maximum  wind  strength.  So,  at  the  moment  the  wind  strength  is _ 6 _ (give  answer 

that  is  appropriate  for  the  current  trial) 

[POINT  OUT  using  pen/curser] 


The  red  line  that  is  pointing  out  from  the  circle  indicates  the  direction  that  the  wind  is 
blowing.  At  the  moment  the  wind  is  blowing  to  the  SE  (say  wind  direction  that  is 
appropriate  for  the  current  trial) 


[POINT  OUT  using  pen/curser] 
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There  is  also  a  second  wind  gauge  below  the  first.  This  second  wind  gauge  indicates  the 
direction  and  strength  that  the  wind  is  forecast  to  change  to. 

[POINT  OUT  using pen/curser] 

The  time  that  the  wind  is  due  to  change  is  indicated  immediately  above. 

[POINT  OUT  using  pen/curser] 


It  is  important  to  monitor  the  wind  forecast,  as  the  spread  of  the  fire  will  change  as  the 
wind  changes.  This  will  also  mean  that  housing  which  has  not  previously  been  threatened 
by  the  fire  may  be  threatened  if  the  wind  changes.  In  the  past,  we  have  noticed  that  good 
Fire  Chiefs  monitor  the  wind  forecast  regularly  and  fight  the  fire  accordingly. 


Do  you  have  any  questions? 


vii)  Flammability 


Now  I’d  like  to  introduce  you  to  another  aspect  of  the  Fire  Chief  program.  In  this  program, 
fires  spread  faster  through  some  areas  than  others.  That  is,  there  are  different  fire  spread 
rates. 


Fires  spread  faster  through  the  bushland  than  through  the  residential  areas. 


These  2  fires  were  approximately  the  same  size  when  they  began.  What  we  will  see  is  that 
the  fire  in  the  bushland  bums  quite  quickly,  while  the  one  in  the  housing  does  not. 


[POINT  OUT  using  pen/curser] 


This  will  be  important  to  remember  when  you  are  deciding  how  to  attack  the  fire. 
Obviously  although  it  is  important  to  attend  to  fires  that  are  burning  houses,  it  is  also 
important  to  put  out  fires  that  are  burning  through  forest  because  it  spreads  so  quickly. 
Likewise,  you  may  be  able  to  leave  for  a  while  fires  that  are  only  burning  through  clearing. 
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viii)  Fire  Warnings 


The  last  feature  of  the  program  that  you  need  to  know  about,  are  fire  warnings.  Fire 
warnings  are  received  from  the  weather  bureau  based  on  weather  conditions  and  the  degree 
of  flammability  of  particular  sections  of  landscape  at  a  given  time.  It  is  safe  to  assume  that 
the  warnings  given  by  the  weather  bureau  are  very  reliable,  and  will  pinpoint  the  time  and 
location  of  a  fire  with  great  accuracy. 

In  the  trials,  fire  warnings  are  represented  by  a  red  cross.  The  location  of  the  warning 
represents  the  location  of  the  future  fire. 


[POINT  OUT  using pen/curser] 


If  you  move  the  curser  over  a  fire  warning,  the  time  that  the  fire  will  break  out  is  shown  at 
the  bottom  of  the  screen. 


[POINT  OUT  using  pen/curser] 


I’ll  now  show  you  how  to  use  the  trucks. 


Restart  demo  trial 


Controlling  Appliances 


i)  Moving  Trucks 

At  the  beginning  of  each  trial,  you  will  have  bases  containing  a  certain  number  of  fire  trucks. 
Fire  trucks  are  used  to  drop  water  to  extinguish  the  fires.  It  is  your  task  to  move  the  appliances 
to  fight  the  fires  that  break  out. 

To  move  an  appliance  you  need  to  use  the  mouse.  Place  the  cursor  on  the  appliance  you  wish 
to  move  and  you  will  notice  that  a  "hand"  appears. 


Click  down  on  the  left  mouse  button,  and  while  holding  it  down,  drag  the  appliance  to  the 
desired  position  and  then  let  go  of  the  left  mouse  button. 


[Demonstrate  how  to  pick  up  a  truck  and  move  it.] 


The  appliance  will  then  make  its  way  to  this  position  -  its  destination  is  represented  by  a 
small  light  blue  square. 

[Move  2  of  each  appliance] 


[POINT  OUT  using  pen/curser] 
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Once  the  appliance  reaches  its  destination,  which  happens  immediately,  you  need  to  click 
on  it  once  to  make  it  start  dropping  water  on  the  fire.  When  water  is  being  dropped,  the 
appliance  will  be  flashing. 


[POINT  OUT  using  pen/curser] 


The  appliances  in  the  fire  station  will  remain  there  until  you  move  them. 


If  you  miss  an  appliance  and  click  on  an  empty  square,  the  computer  will  beep  to  let  you 
know. 


[Demonstrate] 


I  will  now  let  you  have  a  practice  of  how  to  move  trucks. 


Give  mouse  to  participant 
Have  them  move  a  few  trucks 


ii)  Refilling  Appliances  with  Water 

In  order  to  check  what  the  appliance  is  doing  and  the  percentage  of  water  it  has  left,  place  the 
cursor  over  the  appliance  without  clicking  and  look  at  the  information  that  appears  at  the 
bottom  of  the  screen. 

[POINT  OUT  using  pen/curser] 

All  appliances  start  out  100%  full  of  water  at  the  base.  They  can  drop  water  6  times  before 
they  run  out.  When  a  truck  runs  out  of  water,  it  first  tries  to  drop  water  on  the  fire  but  it 
cannot  do  so.  It  will  then  stop  flashing  and  remain  idle.  When  this  happens  you  will  need  to 
drag  the  appliance  to  a  water  source,  where  it  will  refill.  You  do  not  need  to  wait  until  the 
appliance  is  completely  empty  to  refill,  appliances  can  be  refilled  at  any  time  no  matter  what 
the  current  water  level  is.  Once  it  has  refilled,  the  appliance  will  wait  at  the  dam  until  given 
instructions  on  where  to  go  next,  ie.  it  will  remain  idle  at  the  dam  until  you  move  it. 

[demonstrate] 

It  is  worth  noting  that  the  dams  do  not  have  an  endless  supply  of  water.  As  they  empty  they 
will  change  color.  When  they  are  dark  blue,  they  are  full  and  when  they  become  empty  they 
turn  brown.  Generally  speaking  though,  the  dams  do  not  run  out  of  water. 


70 


iv)  Positioning  Appliances 

Obviously,  the  strategic  and  accurate  placement  of  appliances  can  make  a  big  difference  to 
success  in  fighting  the  fires. 


It’s  worth  noting  that  these  trials  are  not  designed  such  that  every  fire  can  be  extinguished. 
Furthermore,  these  trials  have  not  been  designed  so  that  every  house  can  be  saved.  Your 
job  is  to  save  as  many  houses  as  you  can  and  to  attempt  control  of  the  fires. 

Certainly,  the  trials  may  seem  quite  difficult  at  times  in  terms  of  the  number  of  fires  that 
break  out  and  are  spreading.  Just  as  in  real-life,  some  parts  of  fires  must  be  ignored  in 
order  to  save  important  aspects  of  the  landscape  such  as  the  houses,  so  you  have  to 
prioritise. 

Therefore,  when  a  trial  seems  quite  hard  or  even  overwhelming,  remember  you  may  still 
be  performing  extremely  well  if  you  are  saving  houses.  You  receive  100  points  for  saving 
a  house,  so  you  can  be  performing  quite  well  so  long  as  there  are  unbumt  houses  still 
remaining  on  the  screen. 


v)  Number  of  appliances 

As  you  have  probably  noticed,  you  will  need  more  than  one  truck  to  successfully  put  out  a 
fire  front.  Try  and  experiment  during  the  practice  trials  by  using  different  numbers  of 
appliances  to  fight  the  fires. 


If  two  appliances  end  up  on  the  same  square,  it  will  appear  as  if  there  is  only  ONE 
appliance  there,  that  is,  only  one  light  blue  square  will  be  shown  on  the  screen.  This  light 
blue  square  represents  the  LAST  appliance  to  arrive  at  that  location.  If  you  move  this 
appliance  away  to  a  new  location  the  appliance  that  was  hidden  underneath  it  is  re¬ 
displayed. 

Performance  Scores 


i)  Performance  Scores 

The  computer  calculates  performance  scores  for  each  trial  you  complete.  Your  performance 
score  will  reflect  the  value  of  the  landscape  that  you  saved  (emphasise  again  the  importance 
of  saving  houses!) 


Do  you  have  any  questions? 

Close  demo  trial 


Demonstration  =  approx  20 
mins 
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Practice  Trials 

At  this  stage  now,  let’s  clarify  your  role  in  this  experiment.  In  the  following  trials,  you  will  be 
asked  to  assume  the  role  of  the  fire  chief.  As  the  fire  chief,  you  are  responsible  for  a  large 
area  in  Australia.  As  an  example,  imagine  it  is  summer  and  we  are  in  the  midst  of  our  longest 
heatwave  for  years.  We  have  had  several  days  already  where  the  temperature  has  reached  40 
degrees,  so  the  landscape  is  extremely  dry.  Today  is  a  very  hot  and  windy  day.  On  these 
days,  it  is  critical  to  put  fires  out  as  quickly  as  possible,  as  the  longer  they  bum,  the  more 
difficult  they  are  to  contain.  It  is  also  critical,  because  the  weather  bureau  has  forecast  the 
wind  to  change  several  times  today. 

The  area  is  a  mixture  of  bushland,  residential  areas  and  dams.  In  these  trials,  your  goal  as  the 
Fire  chief  is  to  try  and  get  the  highest  score  you  possibly  can  by  saving  as  much  of  the 
landscape  from  fire  destruction  as  possible.  But  remember,  your  score  will  be  calculated 
based  on  the  value  of  landscape  that  you  saved.  So  as  mentioned  before,  the  houses  are  more 
valuable  and  are  therefore  worth  100  points  if  saved  while  the  bushland  is  worth  1  point  if 
saved.  So,  your  first  priority  should  be  to  save  the  houses. 

Just  as  in  real-life,  there  are  different  numbers  of  fire  brigades  that  can  respond  to  a  fire.  The 
brigades  that  do  respond  will  also  differ  in  how  many  fire  trucks  they  possess.  Therefore, 
during  the  trials,  the  number  of  fire  trucks  you  will  have  control  over  will  alternate.  You  are 
free  to  use  the  trucks  in  any  way  you  think  will  get  you  the  highest  score.  That  is,  you  can  use 
as  many  or  as  few  as  you  like  and  put  them  in  any  locations  that  you  want.  Once  again,  you 
will  realise  that  the  fires  will  not  be  able  to  be  put  out  completely,  but  your  job  is  to  contain 
them  as  best  you  can  until  massive  reinforcements  arrive.  Because  you  have  to  deal  with  these 
fires  on  your  own  for  a  while  it  is  imperative  that  you  strategise  about  the  best  way  to  control 
them. 

Also,  you  will  realise  that  you  will  not  be  able  to  save  all  the  houses  in  a  scenario.  Your  job  is 
to  try  to  save  as  many  houses  as  you  possibly  can.  Remember,  you  earn  100  points  for  saving 
a  house,  so  while  a  scenario  may  seem  overwhelming  in  terms  of  the  number  of  fires  that  are 
spreading,  you  can  still  perform  extremely  well  if  you  are  saving  houses. 

Load  up  Practice  Trial  1 

Same  scenario  for  each  participant. 


i)  Practice  Trial  1: 

Here  is  a  practice  trial  for  you  to  do  by  yourself.  In  this  trial  you  will  have  access  to  command 
4  fire  trucks. 

Have  the  participant  click  run  and  start 

Things  to  do  while  the  training  trials  are  running: 

Correct  them  as  much  as  you  can  in  the  early  trials  so  that  they  get  into  good 
habits  without  influencing  overall  strategies,  (e.g.  alert  them  to  appliances  that 
have  run  out  of  water) 

Once  completed,  break  for  5-1 0  minutes 


OK  let’s  try  a  different  scenario. 
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a)  Demonstration  of  SAGAT 

Load  up  demonstration  SAGAT  scene  X2  (have  one  minimised)  with  a 
transparency  aligned  on  screen.  ENSURE  that  you  mark  the  corners  of  the 
landscape  on  the  transparency.  This  is  important  for  scoring  later  on. 


OK,  before  we  start  the  next  practice  trial  I  will  demonstrate  how  to  complete  the  pictorial 
questionnaire.  For  this  questionnaire,  the  computer  will  pause  the  trial  at  any  given  time. 

Ordinarily,  once  the  trial  is  paused,  only  the  basic  landscape  features  will  be  left  on  the 
screen.  The  location  of  fires,  fire  trucks  and  wind  information  will  not  be  shown.  But  for  the 
purposes  of  showing  you  how  to  complete  this  questionnaire,  we  will  leave  these  features  on 
the  screen. 

Following  the  order  of  the  questions  on  the  questionnaire . 

On  the  clear  acetate  sheet  on  the  computer  screen,  use  the  black  pen  to  draw  the  fire  that  is 
threatening  the  most  valuable  landscape  (  out  of  the  fires  that  are  still  burning). 

[Demonstrate] 

Again  on  the  clear  acetate  sheet,  use  the  blue  pen  to  mark  the  location  where  the  next  fire  is 
due  to  break  out. 

[Demonstrate] 


Now  decide  and  mark  on  the  actual  questionnaire  which  of  the  pictures  most  closely  matches 
the  strength  of  the  current  wind. 

[Demonstrate] 


Again  on  the  actual  questionnaire,  mark  the  direction  of  the  current  wind. 


[Demonstrate] 


b)  Practice  Trial  2  with  SAGAT 

OK,  I  will  now  let  you  try  a  practice  trial  that  will  include  the  pictorial  questionnaire. 

Load  up  practice  trial  X2  (have  one  minimised) 

Have  a  transparency  ready  on  the  screen.  Ensure  the  corners  are  marked 

In  this  trial  you  will  have  access  to  command  4  fire  trucks.  When  the  trial  stops,  complete  the 
pictorial  questionnaire. 

Have  the  participant  click  run  and  start 

Have  the  bare  landscape  minimised  ready  to  bring  up  after  pause 

When  done  (close  the  bare  landscape,  press  F4  to  enable  display,  then  Ctrl  R  to 

run) 

Allow  participant  to  finish  the  trial  after  they  have  completed  the  SAGAT 


Once  completed,  break  for  5-1 0  minutes 

Add/subtract  fire  trucks  to  assess  #1  depending  on  their  skill  so  far. 
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iii)  Assessment  trial  #1 

In  this  trial  you  will  have  access  to  command _ fire  trucks. 

Load  up  assessment  trial  #1 

Have  the  participant  click  run  and  start 

Once  completed ,  break  for  5-1 0  minutes 
Assess  optimal  and  adjust  accordingly 

iii)  Assessment  trial  #2 

In  this  trial  you  will  have  access  to  command _ fire  trucks. 

Load  up  assessment  trial  #2 

Have  the  participant  click  run  and  start 

Once  completed,  break  for  5-1 0  minutes 
Assess  optimal  and  adjust  accordingly 


iii)  Assessment  trial  #3 

In  this  trial  you  will  have  access  to  command _ fire  trucks. 

Load  up  assessment  trial  #3 

Have  the  participant  click  run  and  start 


Once  completed,  break  for  approximately  45  minutes 
Final  assessment  of  optimal  and  adjust  accordingly. 

Fill  out  blank  sections  (ie:  no.  of  appliances  in  After  All  Trials  Questionnaire) 

Refresher  Trial 

In  this  trial  you  will  have  access  to  command  (opt _ fire  trucks). 

Load  up  refresher  trial 

Have  the  participant  click  run  and  start 


Experimental  Trials 


Ok,  I  will  now  get  you  to  complete  two  different  trials,  each  with  a  different  scenario  and  each 
with  a  variable  number  of  fire  trucks  available  for  command.  In  these  trials,  you  will  be  asked 
to  complete  the  pictorial  questionnaire  at  several  times. 

Load  the  experimental  trials. 

Have  the  participant  click  run  and  start 
Administer  SAGAT at  appropriate  intervals 
Run  to  completion 


Remember  (for  refresher  and  experimental) 
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For  1  participants,  use  ORDER  1  (MANAGEABLE,  EXCESS),  (MANAGEABLE, 
EXCESS) 

For  2 participants,  use  ORDER  2  (MANAGEABLE,  EXCESS),  (EXCESS, 
MANAGEABLE) 

For  3  participants,  use  ORDER  3  (EXCESS,  MANAGEABLE),  (EXCESS, 
MANAGEABLE) 

For  4 participants,  use  ORDER  4  (EXCESS,  MANAGEABLE),  (MANAGEABLE, 
EXCESS) 

ORDER  1: 

(MANAGEABLE,  EXCESS),  ( MANAGEABLE ,  EXCESS) 


i)  Experimental  Trial  1:  MANAGEABLE  Condition 

Set  up  a  transparency  on  the  screen  ready  for  the  SAGAT.  “This  has  to  be  on  the 
screen.  You’re  not  to  know  which  trial  you  will  need  to  do  SAGAT’’.  Ensure  the 
corners  are  marked  and  note  the  participant  ID  number,  counterbalancing  order 
and  trial  number  all  on  the  transparency.  Do  all  this  without  the  participant 
noticing.  We  don ’t  want  them  to  alter  their  behaviour  in  anticipation  of  having  to 
do  the  SAGAT. 


Load  the  second  experimental  trial  ( MANAGEABLE )  X2  (have  one  minimised) 

Have  the  participant  click  run  and  start 

Administer  the  SAGAT 

(remember  bare  landscape,  F4  and  Cntrl  R) 

Run  Trial  to  completion 

ii)  Experimental  Trial  2:  EXCESS  Condition 

Set  up  a  transparency  on  the  screen  ready  for  the  SAGAT.  Ensure  the  corners  are 
marked  and  note  the  participant  ID  number,  counterbalancing  order  and  trial 
number  all  on  the  transparency.  Do  all  this  without  the  participant  noticing.  We 
don ’t  want  them  to  alter  their  behaviour  in  anticipation  of  having  to  do  the 
SAGAT.  ~  '  '  ~ . * .  ' 


Load  the  second  experimental  trial  (EXCESS)  X2  (have  one  minimised ) 
Have  the  participant  click  run  and  start 

Administer  the  SAGAT  (remember  bare  landscape,  F4  and  Cntrl  R) 
Run  Trial  to  completion 


Once  completed,  break  for  approximately  10  minutes 
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Experimental  Trials  (3  &  4) 

Now  we  will  complete  the  experimental  trials.  As  was  the  case  in  the  practice  trials,  these 
next  scenarios  will  also  have  different  numbers  of  appliances  available. 

Load  the  third  experimental  trial 

i)  Experimental  Trial  3:  MANAGEABLE  Condition 

Have  the  participant  click  run  and  start 
Run  Trial  to  completion 

Administer  the  Individual  Trial  Subjective  Experience  Questionnaire 

Load  the  forth  experimental  trial  (Many  condition)  while  participant  is 
completing  the  questionnaire 

i)  Experimental  Trial  4:  EXCESS  Condition 

Have  the  participant  click  run  and  start 
Run  Trial  to  completion 

Administer  the  Individual  Trial  Subjective  Experience  Questionnaire 

Load  the  second  experimental  trial  (Many  condition)  while  participant  is 
completing  the  questionnaire 

(While  participant  is  completing  the  Individual  Trial  Subjective  Experience  Questionnaire, 
fill  in  the  blank  spaces  on  the  After  All  Trials  Questionnaire  specifying  the  number  of 
appliances  used  in  the  MANAGEABLE  and  EXCESS  conditions  by  each  participant). 

“There  is  one  final  questionnaire  to  complete  which  looks  at  your  experiences  over  all  the 
experimental  trials.  Some,  but  not  all,  of  these  questions  allow  you  to  report  on  any 
differences  you  may  have  experienced  as  a  result  of  change  in  the  number  of  fire  trucks.” 
Administer  After  All  Trials  Questionnaire 

*Thank  subject  for  their  participation  * 


OR 

ORDER  2: 

(MANAGEABLE,  EXCESS),  ( EXCESS ,  MANAGEABLE ) 

i)  Experimental  Trial  1:  MANAGEABLE  Condition 

Set  up  a  transparency  on  the  screen  ready  for  the  SAG  AT.  Ensure  the  corners  are 
marked  and  note  the  participant  ID  number,  counterbalancing  order  and  trial 
number  all  on  the  transparency.  Do  all  this  without  the  participant  noticing.  We 
don’t  want  them  to  alter  their  behaviour  in  anticipation  of  having  to  do  the 
SAG  AT. 


Load  the  first  experimental  trial  (MANAGEABLE)  X2  (have  one  minimised) 
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Have  the  participant  click  run  and  start 

Administer  the  SAGAT (remember  bare  landscape,  F4  and  Cntrl  R) 

Run  Trial  to  completion 

ii)  Experimental  Trial  2:  EXCESS  Condition 

Set  up  a  transparency  on  the  screen  ready  for  the  SAGAT.  Ensure  the  corners  are 
marked  and  note  the  participant  ID  number,  counterbalancing  order  and  trial 
number  all  on  the  transparency.  Do  all  this  without  the  participant  noticing.  We 
don ’t  want  them  to  alter  their  behaviour  in  anticipation  of  having  to  do  the 
SAGAT. 


Load  the  second  experimental  trial  (EXCESS)  X2  (have  one  minimised) 
Have  the  participant  click  run  and  start 

Administer  the  SAGAT  (remember  bare  landscape,  F4  and  Cntrl  R) 
Run  Trial  to  completion 

Once  completed,  break  for  approximately  10  minutes 


Experimental  Trials  (3  &  4) 

Now  we  will  complete  the  experimental  trials.  As  was  the  case  in  the  practice  trials,  these 
next  scenarios  will  also  have  different  numbers  of  appliances  available. 

Load  the  third  experimental  trial 

i)  Experimental  Trial  3:  EXCESS  Condition 

Have  the  participant  click  run  and  start 
Run  Trial  to  completion 

Administer  the  Individual  Trial  Subjective  Experience  Questionnaire 

Load  the  second  experimental  trial  (MANAGEABLE  condition)  while  participant 
is  completing  the  questionnaire 

i)  Experimental  Trial  4:  MANAGEABLE  Condition 

Have  the  participant  click  run  and  start 
Run  Trial  to  completion 

Administer  the  Individual  Trial  Subjective  Experience  Questionnaire 

(While  participant  is  completing  the  Individual  Trial  Subjective  Experience  Questionnaire, 
fill  in  the  blank  spaces  on  the  After  All  Trials  Questionnaire  specifying  the  number  of 
appliances  used  in  the  MANAGEABLE  and  EXCESS  conditions  by  each  participant). 

“There  is  one  final  questionnaire  to  complete  which  looks  at  your  experiences  over  all  the 
experimental  trials.  Some,  but  not  all,  of  these  questions  allow  you  to  report  on  any 
differences  you  may  have  experienced  as  a  result  of  change  in  the  number  of  fire  trucks.” 
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Administer  After  All  Trials  Questionnaire 

*Thank  subject  for  their  yarticivation  * 


OR 

ORDER  3: 

( EXCESS .  MANAGEABLE),  ( EXCESS .  MANAGEABLE ) 

i)  Experimental  Trial  1:  EXCESS  Condition 

Set  up  a  transparency  on  the  screen  ready  for  the  SAG  AT.  Ensure  the  corners  are 
marked  and  note  the  participant  ID  number,  counterbalancing  order  and  trial 
number  all  on  the  transparency.  Do  all  this  without  the  participant  noticing.  We 
don ’t  want  them  to  alter  their  behaviour  in  anticipation  of  havins  to  do  the 
SAG  AT. 


Load  the  FIRST  experimental  trial  (EXCESS)  X2  (have  one  minimised) 

Have  the  participant  click  run  and  start 

Administer  the  SAGAT  (remember  bare  landscape,  F4  and  Cntrl  R) 

Run  Trial  to  completion 

ii)  Experimental  Trial  2:  MANAGEABLE  Condition 

Set  up  a  transparency  on  the  screen  ready  for  the  SAGAT.  Ensure  the  corners  are 
marked  and  note  the  participant  ID  number,  counterbalancing  order  and  trial 
number  all  on  the  transparency.  Do  all  this  without  the  participant  noticing.  We 
don’t  want  them  to  alter  their  behaviour  in  anticipation  of  havins  to  do  the 
SAGAT. 


Load  the  second  experimental  trial  (MANAGEABLE)  X2  (have  one  minimised) 
Have  the  participant  click  run  and  start 

Administer  the  SAGAT X2  (remember  bare  landscape,  F4  and  Cntrl  R) 

Run  Trial  to  completion 

Once  completed ,  break  for  approximately  10  minutes 


Experimental  Trials  (3  &  4) 

Now  we  will  complete  the  experimental  trials.  As  was  the  case  in  the  practice  trials,  these 
next  scenarios  will  also  have  different  numbers  of  appliances  available. 


Load  the  third  experimental  trial 
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i)  Experimental  Trial  3:  EXCESS  Condition 

Have  the  participant  click  run  and  start 
Run  Trial  to  completion 

Administer  the  Individual  Trial  Subjective  Experience  Questionnaire 

Load  the  forth  experimental  trial  (MANAGEABLE  condition)  while  participant  is 
completing  the  questionnaire 

i)  Experimental  Trial  4:  MANAGEABLE  Condition 

Have  the  participant  click  run  and  start 
Run  Trial  to  completion 

Administer  the  Individual  Trial  Subjective  Experience  Questionnaire 

(While  participant  is  completing  the  Individual  Trial  Subjective  Experience  Questionnaire, 
fill  in  the  blank  spaces  on  the  After  All  Trials  Questionnaire  specifying  the  number  of 
appliances  used  in  the  MANAGEABLE  and  EXCESS  conditions  by  each  participant). 

“There  is  one  final  questionnaire  to  complete  which  looks  at  your  experiences  over  all  the 
experimental  trials.  Some,  but  not  all,  of  these  questions  allow  you  to  report  on  any 
differences  you  may  have  experienced  as  a  result  of  change  in  the  number  of  fire  trucks.” 
Administer  After  All  Trials  Questionnaire 

*Thank  subject  for  their  participation  * 


OR 

ORDER  4: 

( EXCESS .  MANAGEABLE),  (MANAGEABLE,  EXCESS) 

i)  Experimental  Trial  1:  EXCESS  Condition 

Set  up  a  transparency  on  the  screen  ready  for  the  SAG  AT.  Ensure  the  corners  are 
marked  and  note  the  participant  ID  number,  counterbalancing  order  and  trial 
number  all  on  the  transparency.  Do  all  this  without  the  participant  noticing.  We 
don ’t  want  them  to  alter  their  behaviour  in  anticipation  of  having  to  do  the 

SAG  AT.  '  ’  '  '  .  '  ~  . " 


Load  the  FIRST  experimental  trial  ( EXCESS )  X2  (have  one  minimised) 

Have  the  participant  click  run  and  start 

Administer  the  SAGAT X2  (remember  bare  landscape,  F4  and  Cntrl  R) 

Run  Trial  to  completion 

ii)  Experimental  Trial  2:  MANAGEABLE  Condition 

Set  up  a  transparency  on  the  screen  ready  for  the  SAGAT.  Ensure  the  corners  are 
marked  and  note  the  participant  ID  number,  counterbalancing  order  and  trial 
number  all  on  the  transparency.  Do  all  this  without  the  participant  noticing.  We 
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don ’t  want  them  to  alter  their  behaviour  in  anticipation  of  having  to  do  the 
SAG  AT. 

Load  the  second  experimental  trial  (EXCESS)  X2  (have  one  minimised ) 

Have  the  participant  click  run  and  start 

Administer  the  SAGAT  (remember  bare  landscape,  F4  and  Cntrl  R) 

Run  Trial  to  completion 

Once  completed,  break  for  approximately  10  minutes 


Experimental  Trials  (2) 

Now  we  will  complete  the  experimental  trials.  As  was  the  case  in  the  practice  trials,  these 
next  scenarios  will  also  have  different  numbers  of  appliances  available. 

Load  the  third  experimental  trial 

i)  Experimental  Trial  3:  MANAGEABLE  Condition 

Have  the  participant  click  run  and  start 
Run  Trial  to  completion 

Administer  the  Individual  Trial  Subjective  Experience  Questionnaire 

Load  the  forth  experimental  trial  (EXCESS  condition)  while  participant  is 
completing  the  questionnaire 

i)  Experimental  Trial  4:  EXCESS  Condition 

Have  the  participant  click  run  and  start 
Run  Trial  to  completion 

Administer  the  Individual  Trial  Subjective  Experience  Questionnaire 

(While  participant  is  completing  the  Individual  Trial  Subjective  Experience  Questionnaire, 
fill  in  the  blank  spaces  on  the  After  All  Trials  Questionnaire  specifying  the  number  of 
appliances  used  in  the  MANAGEABLE  and  EXCESS  conditions  by  each  participant). 

“There  is  one  final  questionnaire  to  complete  which  looks  at  your  experiences  over  all  the 
experimental  trials.  Some,  but  not  all,  of  these  questions  allow  you  to  report  on  any 
differences  you  may  have  experienced  as  a  result  of  change  in  the  number  of  fire  trucks.” 
Administer  After  All  Trials  Questionnaire 


*Thank  subject  for  their  participation  * 
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APPENDIX  F 

PARTICIPANT  DECISION  MAKING  PERFORMANCE  SCORES  AND 
MAXIMUM  NUMBER  OF  APPLIANCES  COGNITIVELY  MANAGEABLE 


Participant  ID 

Perfonnance 

MANAGEABLE 

Performance 

EXCESS 

Maximum  number  of 
appliances  that  could 
be  cognitively 
managed 

1 

64.31 

86.24 

5 

2 

56.33 

55.68 

5 

3 

53.60 

46.71 

5 

4 

60.35 

80.17 

5 

5 

62.60 

75.45 

5 

6 

89.64 

88.34 

6 

7 

62.12 

63.80 

4 

8* 

45.79 

46.61 

4 

9 

96.41 

83.25 

7 

10 

91.01 

85.50 

6 

11 

64.51 

65.88 

5 

12 

81.44 

79.70 

5 

13 

77.82 

65.10 

5 

14 

51.13 

64.07 

4 

15 

87.96 

98.95 

8 

16 

74.81 

80.34 

6 

*Note:  Participant  8  was  excluded. 


